Let's go get it guys!

http://www.majestic12.co.uk/projects...h/download.php


A must have upgrade with important bug related to robots.txt processing fixed and much more CPU friendly archiving that can be controlled using Archiving delays.

Details from history.txt:

Code:
v1.2.1 10/12/05 
      ! Fixed problem with some rare urls resulting in a url bucket load failure 
      + Better strategy to deal with constantly failing buckets 
      ! Content analyser did not care about delays resulting in 100% CPU usage 
      ! Unified all archiving related delays into one called ... "archiving delay" and added it to profiles 
      ! Revised cache flushing logic 
      ! Fixed robots.txt problem when user-agent specified was actually MJ12bot (DOH!)


Major new release with content analysis technology build-in - this reduces size of uploads considerably, generally by at least 50%+. It is highly recommended to upgrade as soon as possible.

Details from history.txt:

Code:
v1.2.0 01/12/05 
      + Smart content analysis (ConAn) technology added to node to greatly reduce size of barrels and increase speed 
      of indexing. 
      ! Improved clustering logic of barrel sorting 
      + Blog XML feeds detection added 
      + Accelerated timeouts applied to extremely slow sites that remain in nearly completed bucket. 
      ! Changed maximum robots.txt delay to be 10 seconds. 
      ! Sorter is more robust now 
      ! Fixed some urls that timed out that were wrongly classified as "Unknown" error type 
      ! Self-restart when timeouts are high changed to at least 60 mins of runtime 
      ! Better filtering of URLs 
      ! Fixed bug in font size parsing 
      ! Fixed problem with non-specified Content-Lengths after content analysis 
      ! Fixed webserver not sending correct authentication header (when auth was enabled) 
      ! Fixed incorrect progress indication for final stage of archiving when archiver is internal (LZMA) 
      ! Fixed potentially fatal SQL syntax error that could force node into endless loop 
      ! Added possibly fatal database error to list of errors that will cause automatic restart of node 
      ! Fixed auto-restarts due to timeouts that happened too early 
      ! Fixed apparent non-completion of some buckets with 1 url left that resulted in bucket load/unloads