Topic for MJ-12 posting only!

[Lagu]

Hej

Wow boys, today we have scored 8.470.000 points and whole 12 members have helped each other to this superb results.

Thank you all from my heart! :thumbleft:

Heroes for a day:

Anih
Cicide
Lagu
Nflight
Mitro
Outlawolf
Evil-Dragon
Jlagnar
Empty_5oul
Ahab3333
Moving fusion

Lagu :D

[BlackAdder]

Ummmm.....left me out :oops:

[Lagu]

BlackAdder

I apologise. I had written down all names but when I should post it I don’t notice I have missed you.

Thanks for helping us in MJ-12. You are one of the heroes too.

Lagu :oops: :)

[NeoGen]

Hehe

That was a distraction there. Lagu said 12 people but only wrote 11 names. :lol:

[BlackAdder]

Thanks guys...no problem, I'm glad to help.I hope something happens to Free DC (nothing bad) so that we can regain our #1.I checked this morning and we still have 12 members crunching including Evil-Dragon, it's good to see him again even if in a limited capacity.
I guess his large numbers come from a connection at his work?? I'd love to have that bandwidth!! :D

[AMD-USR_JL]

Make that 13. I'll try not to download so many movies, and use my internet conecction on Majestic-12.

[mitro]

Good to see we're making a come back! I try to run it as much as I can. I usually have it running unless I'm downloading big files.

[moving fusion]

I would just like to thank AMDave for pointing me in the right direction on how to setup MJ-12.

I am not a big player in the MJ-12 game but i like to do my bit.

[Lagu]

Hi

Another memorable day: 14 members have scored 9.127.904 points today. Can we reach this results tomorrow it should be great.

Thank guys.

Lagu :D

[vaughan]

On the status tab what does it mean:
Uploaded 5 (+7)

[NeoGen]

I think it means that you have uploaded 5 barrels back to base already and have 7 more waiting to be uploaded.

[Empty_5oul]

Vaughan it means exactly what neo said.

If it stops crunching it is probably because you set it to stop when it has 10 barrels (number can be changed under options) all waiting to be uploaded.
This is my problem however mine doesn't upload at all :-(

[Majestic-12]

Woah great achievement guys, congrats!

Empty_5oul: I am going to look in to your problem today, there has to be explanation for this very strange behavior!

alexc

[Empty_5oul]

ok thanks. :D

[gamer007]

Guess whose back! I'm now helping again. I'll have M-12 on almost all nights and whenever bandwidth is free in the day. :)

[Lagu]

Gamer007

Welcome back to our MJ-12 team.
Every member is valuable.

Lagu :D

[Lagu]

Hi

Xgrubbers Kick Ass is only 158.794 points from stop us in a 30 days run.

They are no 2 in todays stats and we no 3.

Let´s do what we can guys.

Lagu :shock:

[BlackAdder]

I just cranked up my settings for MJ12, 100 workers, level 5 compression with winrar, 8 open buckets...let's see what happens. I think my best day with one host has been 40 some buckets, I don't have any more computers on seperate internet connections that I can run this on.Let's go guys...is there any other members who can run this program and help the team??? :D

[AMD-USR_JL]

On the status tab what does it mean:
Uploaded 5 (+7)
To make your node upload those 7 buckets you might want to lower the wait between barrel uploads. Just go to Tools>Options Crawler tab. Here is a pic to help you.http://fatalbits.com/images/aIx67654.png
If you still have a high number you might want to raise the word size or dictionary size in your LZMA compression, but it will increase your cpu usage.

[AMDave]

If the crawler is set to 100% downstream then this sometimes causes probs for the bucket uploads. Due to the upstream and downstream traffic of the crawler, the "control" blocks for the FTP connections are delayed causing timeouts, so sometimes the completed buckets start to "back up" in a queue.

Don't worry about it vaughan. As you are running an off-peak profile like mine, when your client switches from your off-peak profile back to normal ("peak") settings your completed buckets will upload no problems.

For others, if you are running 100% downstream 24-7 and you keep getting more and more buckets waiting to upload, then you may want to insert a profile that backs-off the downstream to around 70% for an hour or 2 while the completed bucket queue catches up.

Happy crawling :D

[Majestic-12]

Glad to hear this gamer007 :)

Actual work on search engines picked up strongly I and hope to reach 1 bln indexed by the end of February, this will be done in installments - 250 mln and then 500 mln or so. Actual fruits of hard labour are about to materialise :)

[Majestic-12]

ok thanks. :D

I've cleaned up database to speed things up and it seems to have fixed Cyber911's node that had similarly weird upload problems, so he is okay now and it would be good if you could check if your node can now upload - if it still won't then please let me know and leave it running - it will stop crawling and thus use no resources, but it will continue to try to upload - this will give me chance to debug things on a low level here.

cheers

alex

[Empty_5oul]

yeah that sorted it!!!!!

thanks.
-- tomorrows score should be more impressive, lol.

[Majestic-12]

Excellent - if I kept count that would have probably been bug number in region of Googol or something ;)

[Lagu]

Hi

First a bad news:

Free-DC have stopmed us and is now no.1 owerall and we are a good 2.
Py 222 is hopeless and limitedless. He alone scored 8.970.000. Free-Dc should give him a Gold medal.

Good news:

Very, very good done guys!

We have reached the old record of 9.127.904. We hit 9.281.166 points and whole 15 members have helped done this results.

Thanks everyone!

Lagu :D

[Lagu]

Hi guys

X Grubbers kick ass is now number 2 in the stats and AMD Users number 3. Now we must keep the 4th rank!

Lagu ;) :)

[Empty_5oul]

in the process of fixing my upload i clicked "migrate node" how do i un-migrate it????

[BlackAdder]

X-Grubbers are #2 in total daily stats, we are still #2 overall :D
Lets go TEAM!!!

[AMDave]

It appears that we have passed the 500,000,000 URLs Milestone.

YEEHAAA !!!
:hello1: :hello1: :hello1: :hello1: :hello1:

[Majestic-12]

Congrats! That's more than 1/10th of last year's Google (4 bln pages) :)

alexc

[Lagu]

Hi

Guys, are you redy for a fight :?: :?:

I will tell you our points overall is 552.292.221
X Grubbers Kick Ass have 519.232.834 a differense of only 33.059.290 points.

Today we earned 3.518.000 points.
X Grubbers earned 7.360.000

They takes everyday a large step and will soon come closer to us and to the end stomp us.

We must run MJ-12 continuously and at least 13-14 members must crunch at the same time. We can not run 2-3 days and then disappear and leave 6-7 members to fight.

How vill we classify this project on a scala from 1-10 where 10 is higest priority and 1 the lowest?

Lagu :) ;)

[BlackAdder]

I'm doing what I can, I've got a Cisco cable router and modem combo coming next week...I hope that since this unit has a real firewall, processor and it's own ram it will help my throughput quite a lot. Thats if my provider lets me use it, it's not on their list of preferred modems. :D
I'd like to put out what Evil Dragon does :lol:

[Lagu]

BlackAdder

I have notised how well you are cruncing MJ-12. If we all have Cicides or Evil-Dragons capacity we chould not be number 3 every day in the daily stats.

I don´t know if many have a limit on their broadband but I have run MJ-12 long and newer heard from my ISP even if there have been many hundred MB used.

Lagu ;)

[BlackAdder]

THANX !!! Keep crunching team...x grubbers is very close on us now. Has anyone noticed that when running MJ-12 and D2OL at the same time that MJ-12 stalls out?? Could it be they both use Java and maybe D2OL's priority is set higher so MJ-12 has to wait?? any guesses? :)

[Majestic-12]

MJ12 uses .NET rather than Java, but if you run it concurrently with a high CPU usage program like SETI @ HOME, then please assign higher priority to MJ12 - since MJ12 uses less CPU than everything else it is better off overall give it as much CPU as it can get (minority of it), but the rest will be used by whatever else you running - you can do it using priority of MJ12 in Options->Misc :)

[NeoGen]

By the way... any plans for a release (maybe even just a special one) compiled against .NET 2.0?
:)

[KAMCOBILL]

http://www.amdusers.com/modules.php?name=Forums&file=viewtopic&t=1848&start=15

[Majestic-12]

By the way... any plans for a release (maybe even just a special one) compiled against .NET 2.0?
:)

Yep - very shortly, as soon as we hit 1 bln pages I am going to revamp client, definately try .NET 2.0 as I think it will be using less memory.

This is not far now - we hit 200 mln already and next week I will do another update, and perhaps one more or two at the worse updates after that will lead to 1 bln pages - not a small search engine! :)

[KAMCOBILL]

We're Ready.

[BlackAdder]

Hay Guys, I've got a new router working and my MJ12 output is way up, how bout some more help tonight and Sunday and lets see if we can stomp the X Grubbers :D we did 4,090,000 today so I know we can do better tomorrow , we got within 900000 of the X Grubbers today!! Great job all.

[Lagu]

Hi BlackAdder

I’ve noticed you are on the top of our team crawling 1.500.000 Url`s and this is a great results. You are an AMD User hero. I have paused MJ-12 since yesterday due to too slow connection speed. Your posting make me so inspired so I rebooted the computer and started MJ-12 and the connection speed get better, little over 1 Mbits.
Thank you for your positive posting.

Lagu ;) :D

[BlackAdder]

Thanks Lagu, I thought I'd lead the charge and try to dislodge the X Grubbers from yet another win. We were ahead today for at least an hour and it looked like we might pull one out but I think they got more guys on line and we were out scored...it's almost 6 pm and we scored 6,300,000 so the team did have a winning day :D
I,ve been trying to increase my output since Evil- Dragon's output went down , thus the Cisco cable modem I bought but it wasn't compatable with the cable provider :-(
My new weapon is a Linksys BEFSX41 router , the thruput of this unit must be 10 times greater than my last router and I bought it on Ebay for $20 some dollars...oh well lets keep fighting TEAM !!!

[Lagu]

Hi

I notised we have got a new member 11/2 kl. 23.41.29 and his nickname is shett and is from Great Britannien.
It is wery good when peoples join us and I wish him good luck.

Lagu :D

[BlackAdder]

WOW...it looks like we whipped the X Grubbers two days in a row...way to go TEAM !!!
:D :D :D :D :D :D :D

[Lagu]

Hi

KAMCOBILL in X-Grubbers is out of the race because his ISP have removed/closed his account due to too large use of bandwidth whish explain why we can outrun this Team.
He have only got 10.000 points so far.

Lagu :)

[cicide]

My network provider (Charter Cable) has comepletely hosed up my service. As soon as I can afford to have them back in here (they killed it for 4 days outright last time they tried to fix it) to fix it, I'll be back to 1M per day. Until then I'll be doing not much at all.

[BlackAdder]

OUCH....too bad for KamcoBill....so that is why we are catching them....I better watch my cable modem service too cause I'm sure I'm not on an unlimited account. But until then...GO TEAM!!! :D

[Lagu]

Hi

Yes, this is to bad. Nothing we wich our friend :!:

Reading the forum of Majestic-12 shows KAMCOBILL use his 56 Kpbs modem so far :shock:

Lagu :)

[AMD-USR_JL]

The same kind of thing happend to me. My ISP capped everyone on my line, cause i had used so much bandwith for such a long time. The day is actually recorded in MJ12 Hall of Fame (http://www.majestic12.co.uk/images/misc/records/70mln.png). I think i was 13th, which might have been an omen now that i think about it. The connection was very very slow for almost a week. It got worse, they had capped it cause they were trying to find the source. They eventually found me and my modem couldn't connect to the network for a whole day!!! I think this is really gay cause on there contract thing they say you get to use a 24/7 5mbit/sec of download and 768kb of up for $XX/month, but if i ever use it constantly they do that to me. I am too scared to start it again. I will try to build up my courage and run it at 1mbit/sec which would be a fifth of its capacity, i don't think they will mind that too much.

[NeoGen]

You should review that contract and see if there are in clauses in small fonts that you may have missed. Maybe monthly bandwidth limits or something like that?
In case the contract really says you can use the full pipe 24/7 (and there are no small font clauses) and they still don't let you, then I guess you should complain...

[Majestic-12]

I am too scared to start it again. I will try to build up my courage and run it at 1mbit/sec which would be a fifth of its capacity, i don't think they will mind that too much.

I am sorry to hear about your ISP being arsey about service that you pay money for :(

Some ISPs seems to be more sensitive (or one my say unreasonable) than others - if you do get a warning from ISP then don't hestitate to reduce crawling 5-10 times - this is likely to be enough to put you under their radar, its better 512kbits crawling than 0kbits crawling.

I hope there will be influx of people to the project and this would allow individuals to crawl less - but due to higher number of people crawling will still be high, yet ISPs won't be able to turn on individuals since on per person basis traffic won't be high.

What really cheeses me off about ISPs is that they allow BitTorrent traffic, Emule etc - they now account for like 30% of Intrenet traffic, and here its a legit application working towards rather useful goal :(

Not happy :shock:

[Majestic-12]

Just to let you all know - we are talking about interesting idea how to reduce bandwidth usage considerably - see thread here (http://www.majestic12.co.uk/forum/viewtopic.php?t=1396), it seems to work but sadly there is a cost of 25 GBP per year :shock:

[AMD-USR_JL]

What really cheeses me off about ISPs is that they allow BitTorrent traffic, Emule etc - they now account for like 30% of Intrenet traffic, and here its a legit application working towards rather useful goal :(

Not happy :shock:

My ISP already filters my emule, bittorent, limewire. It is really gay and i hate it. One dummy who was using my isp got caught, he wasn't using any ip blockers which makes him a dummy. The mpaa sued cox, so now they filter everyone's p2p traffic or just traffic in general like mj12.

[NeoGen]

I don't know how ISP's filter p2p traffic nowadays, but if it's still by blocking certain port numbers, then at least with emule you're in luck because you can manually assign (almost) any port number you want.
They can't block *all* ports now, can they?? :lol:

[Majestic-12]

now they filter everyone's p2p traffic or just traffic in general like mj12.

That's taking the piss really because MJ12 traffic is perfectly legit and its also web traffic :?

I can understand an ISP not happy about bittorrent, but MJ12 traffic is not only legit but also accounts for a very very small %-tage of overall traffic :?

[AMD-USR_JL]

I don't know how ISP's filter p2p traffic nowadays, but if it's still by blocking certain port numbers, then at least with emule you're in luck because you can manually assign (almost) any port number you want.
They can't block *all* ports now, can they?? :lol:

I've tried almost every port and they block it in a different way. I can download as much as i want, but i can only upload to other cox ips or someone who is uploading to me. This ok on emule but on a bittorent site where you have to have a 1+ ratio, its not so good.

[Ototero]

Did anyone notice that we quietly took 2nd place back :D :D :D

[BlackAdder]

Yep...earlier this week we took back 2nd.Let's see if we can hold on to it...GO TEAM !!! :D We had a 2 million point lead the last time I checked.

[Lagu]

Hi BlackAdder

Where are you? I can´t se you crawling today. Have you any problems with your computer or connection?
I have fired up my computers again after I have run MoneyBee and uFluids under the weekend and I hope I can crawl 1.600.000 urls.

You are the only I can talk with abouth MJ-12 because other who are running this project not use to be here.

It is high time to take our 2th place back or how?

Lagu :D ;)

[Nflight]

Since everyone else is commenting, I will too. I too have had a spat with the Provider. So I removed the program and refuse to tolerate the usage although I was doing well when the fan hit the poop per say!

If your wondering how they adjust and monitor your traffic, they use a Frame Relay system which they can adjust and alter the incoming and outgoing usage montioring down to every last bit. Limitations are also easy to apply.

With some creativeness I removed the limitations on my connection and was zooming at 40mbs download for a short time, till they noticed. So here I am not running Majestic anymore.

I still feel for the team effort and support everyone who is still in there, I recommend not removing the limiters from the circuitry of your DSL connection without authorization.

[Lagu]

Nfligth

Sorry to hear but I understand you. You have done well so long as you was running MJ-12 and you shall have many thanks for that.

Lagu :D ;)

[Evil-Dragon]

Once my furniture arrives I will be back to crawling at full speed. Also just invested in a nice 4200 X2 with 2GB of RAM, going to build it tonight. Watch out I will be back soon :D

Also we lost our 2nd place again...

Finally, Alex is going to be incorporating some new compression this week-weekend which reduces compression times down to a mere 10 seconds on a 1.8Ghz Mobile Sempron AMD processor. I have tested this on my laptop and it is very fast and also reduces barrel sizes by up to 30%, which is very good!

Keep watching MJ12 for further developments.

[NeoGen]

At this rate of improvements we'll soon be uploading just a few hundreds of kilobytes per barrel. :)

Great work Alex!! :D

[Majestic-12]

At this rate of improvements we'll soon be uploading just a few hundreds of kilobytes per barrel. :)

Thanks and I wish it was possible! :)

In some sense it may well be - when smart recrawling is implemented, this is rather tricky but shortly it will be good time to deal with it - when implemented it should ensure that only changed pages are uploaded - this should cut down upload sizes further, possibly by 2-3 times - I'd say if its 1 MB or less then its really good enuff!

[Lagu]

Evil-Dragon

It sounds good and I will wait to the day you are ready to show us what your newest computer can do.

Lagu :)

[Lagu]

Majestic-12

Great if it will work as you have planned. I guess you will release a new agent soon?

Lagu :D

[Lagu]

HI

Great news. We have taken back ur 2th rank total again after I use my artillery!

Lagu :D

[Majestic-12]

I think new beta will be out in couple of days - it will contain support for multiple upload targets which will increase overall server capacity, and I think new compression will appear early next week - its a serious question that needs good testing, don't want to make stupid mistake that will result in data compressed that can't be uncompressed :?

[BlackAdder]

Hi all.....sorry but I have been dedicating my resources to other DC work this week :cry:
It's back on tonight and should be able to run it until this weekend, I will be out of town Sat. and Sun. but will leave it running. I get a fault on my second computer with MJ 12 about being unable to load resources or something like that after it runs maybe an hour...any ideas what that means???
AS always...GO TEAM !!! :D

[Majestic-12]

I get a fault on my second computer with MJ 12 about being unable to load resources or something like that after it runs maybe an hour...any ideas what that means???

No idea right now, do you want to post some details in bugs section in our forum? I will certainly look into it!

[BlackAdder]

Sure , I'll see if it happens again and write down exactly what the message box says. I'd be glad to add it to the MJ12 forums...Thanks !! :D

[BlackAdder]

Just added the fault under "MJ12 Fault" heading in the Bugs thread...Thanks !!! :D :D :D

[Evil-Dragon]

My new machine is now built, i just need to install Windows 64 bit edition on to it which i will do tonight. Spec as follows:

AMD X2 4200+ (2.2Ghz)
Corsair 2GB RAM
Asus A8N-SLI-DELUXE MB
Sapphire Technology ATi X1900XTX 512mb PCI-E
Western Digital Raptor (10,000 rpm) 74GB drive
Western Digital 300GB drive
Western Digital 400GB drive
Western Digital 250GB drive (as you can i will have no disk space problems
:lol: )

Thats the spec at the moment, i won't boather going into sound cards and PSU's. :)

[NeoGen]

That's a really powerful machine you built there! :)

Are you going to run any other projects on it besides MJ-12?