Usage

Setup

Ebot options are in

• ebot.app file
• sys.config

You should add or change options only in ebot_local.config; or adding them to the command line (adding at the end of the line of start.sh something like -ebot Param value)

Start

Start Rabbitmq wiith rabbitmq_server/sbin/rabbitmq-server
Start Couchdb/Riak server
Start statistics with ebot/utils/update_rrd.sh
Start Ebot with ebot/start.sh

Eunit tests

Start Ebot with ebot/start.sh and then run ebot_test:test()

Managing Crawler

• http://localhost:8000/crawler/start
• http://localhost:8000/crawler/stop
• http://localhost:8000/crawler/add_url?url=http://www.redaelli.org/
• http://localhost:8000/stats/show

See also src/ebot_test.erl for details

Managing Workers

• http://localhost:8000/worker/html/start/?depth=N&tot=M
• http://localhost:8000/worker/html/check_recover

• http://localhost:8000/worker/web/start/?depth=N&tot=M
• http://localhost:8000/worker/web/check_recover

AMQP

You can monitor how many urls are wating in MQ server with

http://localhost:8000/stats/show

or running

rabbitmq_server/sbin/rabbitmqctl list_queues

the output will be something like

ebot.url.refused 37
ebot.url.new.1 8830
ebot.url.new.0 3238
ebot.url.new.2 5683
ebot.url.new.3 0
ebot.url.new.4 0

In ebot.url.new.0 there will be urls with depth == 0 (ie http://www.redaelli.org, http://www.redaelli.org/index.html, …)
In ebot.url.new.1 there will be urls with depth == 1 (ie http://www.redaelli.org/matteo/, http://www.redaelli.org/matteo/index.html, …)

Other web commands

• http://localhost:8000/help
• http://localhost:8000/stats/show