active workers dropped to a really low number

[taters]

I have my workers max set at a 100 for faster crawling.
Right now I have 5 buckets in queue, and all of them are near
completion, i.e. less than 500 left on each. However, they are taking
a very long time to complete as they all show only 1 worker for a total
of 5.
Any suggestions?

[Empty_5oul]

they go much slower at the end than the start as there is less left. Leave them for a bit and see if they complete.

[Lagu]

Taters

You have too many workers working. Reduce it to 50 and wait and see.

Also an important things:
How many percentage is your Successes? Under 80 % isn´t acceptable.
If it is to low try reduce workers to 25.

Percentage under Retries?

If there is a high percentage migrate the node for sending the files back to the server and you receive new frech URL´s.

Go to "Tools" click on "Migrate node".

Lagu

[Lagu]

How change Workers:

Rightclick on MJ12 go to profiles then to Crawling. There you have:

Maximum number of ascync workers. Set it to 50. 25 or 10 and test how it works.
Maximum URL bucket: Set it to 15, 25 or 30.

Restart the node.

Important: If you migrate the node your maximum URL bucket will be empty. Change it to whatewer you want. Then restart the node.

Lagu ;)

[Lagu]

Hi taters

If you have more questions or problem join Majestic-12 forums. The moderator/admin is Alexc and he is wery fast to ansver every question we have.

Here is the link: http://www.majestic12.co.uk/forum/

Lagu ;)