The demo is a simple crawler that visits a few sites and then generates a report. The crawler is written in Python and uses Selenium to drive the Tor Browser.
- Install the Tor Browser.
- Run the crawler.
- Run the report.
Use with the container:
docker run -it --rm -p 5901:5901 -v "${HOME}/src":/src excitingtheory/kalilinux-xvfb:torbrowserStart the VNC server:
/opt/start-vnc-server-once.shWith Finder open as the main app. Press cmd-k the "Connect to Server Dialog" will open. Enter vnc://localhost:5901 and the password guestpas when prompted.
Dialog can also be found in the finder menu: Go -> Connect to Server
Download and install a vnc client like TightVNC and connect to localhost:5901 and the password guestpas when prompted.
Go to the correct directory:
cd /src/tor-driver-pythonRun the crawler:
python3 crawler.pyRun the report:
python3 report.pySchema:
{
"name": "Meow",
"file": "./results/meow.json",
"search": "Meow ",
"orig": "Meow\n",
"num": "",
"artifacts": [
{
"data": "Jump to content",
"link": "https://en.wikipedia.org/wiki/Meow#bodyContent",
"parent": "https://en.wikipedia.org/wiki/Meow"
},
{
"data": "",
"link": "https://en.wikipedia.org/wiki/Main_Page",
"parent": "https://en.wikipedia.org/wiki/Meow"
},
{
"data": "",
"link": "https://en.wikipedia.org/wiki/Wikipedia:Contents",
"parent": "https://en.wikipedia.org/wiki/Meow"
}
....
]
}