PDA

View Full Version : Majestic-12 Distributed Search Engine



Evil-Dragon
10-13-2005, 05:40 PM
Hi vaughan, i have recieved your e-mail and have agreed to give over the rights to the team on MJ12.

Please rejoin the group and i will transfer the rights.

Also can you add Majestic-12 onto your list of DC projects?

Best Regards
Matthew
Evil-Dragon @ Majestic-12

Empty_5oul
10-13-2005, 06:24 PM
many thanks. :D

NeoGen
10-13-2005, 06:51 PM
Cool! Grub's gone, but we got a new toy to play with :lol:
Here's a linky for anyone that wants to give it a spin... http://www.majestic12.co.uk/
Warning to everyone: It's a big bandwidth hog!

Hey evil-dragon, will you stay and crunch with us? :)

Lagu
10-13-2005, 07:22 PM
Evil-Dragon

Thank you for your reply to my e-mail. I have read it and then checked 12-site. To my surprise, I noticed 2 AMD-team. I didn’t like it. I am used to join AMD users with our logo. ;)

I wish to say: very, very welcome to us. We other members are happy to have you among us :wav:

We are thankfully for every new member and especially if they also join our forum as you have done. Well done! All experience is welcome. We all never will be fully trained. Needless to say, many of them as join our projects are far too silent and anonymous. :cry:

What is your knowledge and experience about Majestic12? Mostly of us are newbie’s in this project. :!:

I myself joined Majestic12 Monday 10 in the evening and have already reached over 2.000.000 url´s. I’m surprised that it goes so rapidly. I have 2 agents running simultaneous on 2 AMD`s. :)

I will join our team soon. Right now, I have problems logging in on their team-site. I have E-mailed Alexc for help solving this.

Lagu :D

Evil-Dragon
10-13-2005, 07:24 PM
Having a few probs transfering the admin rights at the moment cause of a small bug, but Alex should fix it soon.

Yeah i was gonna create a new team but shall probably just stick with the "AMD Users" team now.

Once the transfer problem is fixed I shall post again.

In the mean time can anyone that joined the "AMD-Users" team join back with the other one since otherwise the you will lose any team stats collected once vaughan disbands the "AMD-Users" team.

Lagu
10-13-2005, 07:32 PM
Evil-Drogan

I apologice. I haven´t read NeoGen´s comment asking if you should join us. I though I was to rapid to say welcome.

Lagu :oops:

Evil-Dragon
10-13-2005, 07:44 PM
Who should I transfer it to?

Can't transfer it to vaughan yet because he's in the other team.

There's a choice of:

Beerknurd, bwhite, ototero

or shall I just wait?

Empty_5oul
10-13-2005, 07:50 PM
any of them would be fine BUT if you could wait till australia is up then Vaughan will come on and sort it out (him being our admin means he would be the best option)- if you say there are problems with transferring owner rights anyway waiting a few hours shouldn't matter.

cheers.

Evil-Dragon
10-13-2005, 08:01 PM
What is your knowledge and experience about Majestic12? Mostly of us are newbie’s in this project. :!:


I know a lot about MJ12, been with them for just over a month. Typed and wrote the first draft of the help documentation for it and been helping out on their forums. Done over 62 million urls at the time of writing.

Alex (alexc) the owner/admin/programmer of MJ12 knows everything though so if i can't answer then i'm sure he definatly can.

NeoGen
10-13-2005, 08:21 PM
I would just like to add that if people reading this thread that if you don't find teams anywhere, it's normal... they're not in plain view on the site yet.
To get into a team, one must go here --> http://majestic12.kicks-ass.org/
Once there, login with your Majestic-12 info, and go to the teams section, and join a team.

And if anyone has Norton Internet Security, disable it before you go to that site. I have Norton and wasn't able to login until I disabled it. After exchanging some posts with Alex on the forums, he pointed that it could probably be Norton blocking it someway... and I tried, and it really was.
Don't know if other firewalls also have this behaviour but Norton Internet Security 2005 is guaranteed... :-(


p.s. - And welcome aboard Evil-Dragon! :D If you would like to try other projects too, we got plenty to choose. Just look at our stats ;)

Ototero
10-13-2005, 08:35 PM
Hey Evil-Dragon,

Thanks for allowing us to join/merge.

I'll create the stats for us over the weekend. Individual stats could be a problem bcause selecting AMD Users to see the individual scores just shows the map.

But I have to look there 1st, to see ALL the members, then check another page to fetch their scores.

I'll try to sort something out.

EDIT

Couldn't wait for the weekend, so the team stats are done. The individuals can wait.

END EDIT

vaughan
10-13-2005, 11:41 PM
NeoGen,
I need you to leave AMD-Users and join AMD Users. Then as the only member of AMD-Users I will be able to disband it and then join AMD Users yself. Then E-D can transfer admin rights to me. Phew! :!:

Vaughan

NeoGen
10-14-2005, 12:13 AM
Oops... sorry. Forgot about that. Actually with all the switching back and forth I didn't know anymore where was I... I thought I was in the no-team. :lol:
But I'm out now. Old team is ready for disbanding.

vaughan
10-14-2005, 06:03 AM
Done.

NeoGen
10-14-2005, 11:39 AM
Vaughan, all that's missing now would be moving the Grub forum section down to the finished ones and open up a new section for Majestic-12 :)

Evil-Dragon
10-14-2005, 12:06 PM
Rights transferred, enjoy! :)

vaughan
10-14-2005, 02:16 PM
Forum thread created.
Thread moved from Grub to Majestic-12.
spjeff69 and vaughan as Moderators.
Misc projects sorted in alphabetical order. (Will try to tidy the others when I get time).

Ototero
10-14-2005, 02:42 PM
All stats are up and running :cool:

NeoGen
10-14-2005, 02:43 PM
Vaughan, now that I see the misc section sorted alphabetically (very nice!), I got a suggestion...
Wouldn't it be better to switch some of the projects names... like for example instead of "Boinc - Burp", "Boinc - Primegrid", etc, putting the name of the project first and the reference to boinc afterwards? Something like... "Burp - Boinc", "PrimeGrid - Boinc", and etc so that the projects would really be sorted by name.

vaughan
10-14-2005, 02:51 PM
Yes - will do when I can (its 1:50 am, time for bed)

NeoGen
10-15-2005, 07:02 PM
Our old team still appears in the stats... will alexc remove it one of these days?

Empty_5oul
10-15-2005, 07:27 PM
he should make it automatically disappear if no members.
As more poele join and it grows this will be a increasing problem as people constantly jump, create and delete teams.

Anonymous
10-15-2005, 08:21 PM
Hi guys,

Thanks for joining MJ12 project :)

When team was abandoned nobody can join it, but to be sure I deleted that other AMD team.

If you have any such requests then don't be shy to post them in MJ12 forum.

cheers

alex

NeoGen
10-16-2005, 01:33 AM
Hiya Alex

Don't know if you'll come by again to see this, but great to have you around here with us! :)
I didn't ask in the MJ12 forums because it's a minor thing, the old team there doesn't hurt anyone. And I'm betting you got a big to-do list with much more urgent things than this. I was just wondering if you had to remove disbanded teams manually from the stats or if it was an automatic process. At first I thought it would disappear automatically, but time passed and it was still there... lol


Now speaking to everyone...
New version of the MJ-12 app is out! Plenty of bugs were fixed and a new compression method was introduced, LZMA.
I wonder if this is better than RAR... could anyone test it?

Evil-Dragon
10-16-2005, 10:44 AM
It is better than RAR with compression however it can take a lot longer to compress barrels. It's mostly there for new MJ12 users that don't have WinRAR installed or can't/won't install it.

It will help alot with cutting down bandwidth server side since GZ was taking up nearly 50% of Alex's bandwidth :shock:

NeoGen
10-16-2005, 11:23 AM
Cool! I don't mind waiting a little more on compression if that means smaller uploads. Bye WinRAR! :)
By the way... any hints on what's the best settings for word size and dictionary size? I take it that the dictionary size is as RAR was too, the bigger the better, but word size I don't know.

Anonymous
10-16-2005, 02:50 PM
I am always around - thanks for creating special category for Majestic-12, it is pleasing to see this project among other older more established ones :)

Evil Dragon is right -- LZMA is better than RAR in terms of compression (if you increase dictionary size and words size - see FAQ entry on our forum), but its slower for compression -- however you guys have very fast processors so you might actually be better off trying it.

From my point of view I wish everybody used LZMA because I have native implementation for decompression which is very very fast, faster than RAR. :)

Remember - to switch off RAR you just need to disable it and LZMA will take over. In my tests the following settings always generated smaller data than rAR:

1) Dictionary size: 4 MB or higher (dont overdo it - 10 times more memory than dictionary is used during compression)
2) Word size - 32 or higher.

I will also add another parameter next time -- algorithm type, I used the fastest at the moment but it won't compress as well as 2 others, I did it because I felt it was slow enough already, but again you have very fast processors and I always assume rather old CPU.

NeoGen
10-18-2005, 04:27 PM
Vaughan, since we don't have a team logo small enough to fit at the team site on Majestic, how about putting this little one we got on the mainpage?
http://www.amdusers.com/danny/amdaffiliate.gif

Could do the trick until someone designed us a symbol small enought that we could put there :)

vaughan
10-18-2005, 07:55 PM
OK, its added to the appropriate team admin section in MJ12's site but it didn't display when I looked. Strange.

Anonymous
10-19-2005, 12:02 PM
[quote="NeoGen"]Vaughan, since we don't have a team logo small enough to fit at the team site on Majestic[/img]

I can allow bigger image sizes if necessary. :)

btw, anyone seen Beerknurd around? He does not answer on his email :(

Ototero
10-19-2005, 05:25 PM
Yes, now that you mention it, he hasn't been around for a day or so.

Beer, are you vacationing ;)

Evil-Dragon
10-19-2005, 06:25 PM
Btw I found some pretty good compression settings if you're interested in using LZMA.

Dictionary size: 4-6mb
Word size: 40 bytes

Now set the archiving delay in Options, Crawler to 10

This will use 80-85% CPU but give some good results. Compression takes 3-5 mins depending on barrel size. Not bad and compression is good.

You can probably increase the word size/dictionary size but you will probably get minimal savings on the compressed size.

Ototero
10-19-2005, 07:11 PM
Evil,

I just made those changes, let's see what happens. I'm staying in a B&B in Milton Keynes using their bandwidth :lol:

Empty_5oul
10-19-2005, 09:07 PM
if you want to tell him something (beerknurd) PM him on these forums.

then when he checks that will be one of his first priorities.

Lagu
10-19-2005, 09:27 PM
Hello

The Ethernet card as is in my AMD64 seems to be corrupt. I can´t obtain a address automatically. I have tried all and should replace this card but to my surprise I found a little component integrated with the wotherboard. As it not was enough, a large fan of 12 cm is mounted and I couldn´t remove. Today Ã*sn´as yesterday when a card was easy to remove or replace. I must have the computer to my reseller and hear what is wrong.
All Majestic 12 chrunching and OGR is stopped.

Lagu :cry: :cry: :cry:

Empty_5oul
10-19-2005, 09:54 PM
try putting in a new network card and use that. turn of the onboard network and it should be fine. replacing the motherboard is a bigger (prob. more expensive job).
where exactly is the fan?

Lagu
10-19-2005, 10:07 PM
Hej

It is placed in the middle of the motherboard. And the compunent is difficult to remowe. Little place for my fingers. As all other it is plugged hard. The componet is as 4 and 6 sugarbits high togheter.

Lagu

Beerknurd
10-20-2005, 02:07 AM
oopsie..... problem fixed. Sorry Alex!!!!!! I upgraded.... :oops:

Anonymous
10-20-2005, 01:39 PM
oopsie..... problem fixed. Sorry Alex!!!!!! I upgraded.... :oops:

No probs Beerknurd -- now you should be uploading much faster than before
as LZMA or RAR produce much much smaller barrels: this saves bandwidth for everybody :)

Beerknurd
10-20-2005, 10:15 PM
Why is Lagu not on our Majestic team???? We would be alot more ahead if he was......

NeoGen
10-20-2005, 10:28 PM
Good question... what happened to Lagu? He was on our team...

vaughan
10-20-2005, 10:38 PM
I saw a post somewhere that he was having network card problems and was taking his computer(s) back to the shop to get fixed. I think there was a fan on the motherboard that needed replacing too.

Lagu
10-21-2005, 01:22 AM
Hello

I´m alive and nedless to say, I got some failures on my Ethernet Card on my AMD64. I got wrong IP-addresse and have phone my ISP and they said the connection between their server and my modem is ok.
There is somthing with Web proxy. Windows say it is not configured :?:

My other computers is full working. However, I ran node 12 on my Athlon and it went fine. Suddenly it was stop :!:

I checked Ewents Wiewer and there i found this message:

"TCP/IP has reached the security limit impossed on the number of concurrent TCP connect attempts".

This is very danger because if you got this you got to many timeouts and soon it is stop. This "security" is typically for Win XP SP2. It slow down crawling and cause too many timeouts. You can not download or upload anything. After that i got many connection errors when I ran node12 on my Athlon. The solution for this is on this link:

www.speedguide.net/read_articles.php?id=1497

There you can find a patch as remove the limit on TCP connection attemps.

The patch is Event ID 4225 Patcher v2.11

I had to join our group but fail to log in. Then I got problem and now I´m running node 12 on my Athlon but the nev version of the node seems slower than before. On one hour I have crawled 6.769 urls. It is far from the first version of this node. I´am out of the play and lose my rank.
But i hope you other will crunch fore me ;)
As soon my Ethernet card is changed or configured i will come to you all other with open arms.

Lagu :cry: :cool: :)

Lagu
10-21-2005, 06:16 PM
Why is Lagu not on our Majestic team???? We would be alot more ahead if he was......

Yep, Beerknurd, now is I among you other in Majestic12. I got eight millions five hundred seventeen seven and nine hundred one (8.577.991) crawled url`s in my backpack. It feels fine to come to AMDusers team. I was waiting regarding to the horrible trouble we had with two AMDusers. I though, when we have our logo in our team, I should join. But I fail because there was any faulty when many of us should use our username and it wasn’t accepted. Alexc come to my help and late this evening I got a success. We should go from 43.000.000 to 51.500.000 and I’m glad to get a boost. My Athlon 5126 EA is running node 12 night and day. But, I must look after too much errors and the percentage of successes won’t be to low. About 80 % is good I think.

Lagu

NeoGen
10-21-2005, 07:12 PM
I think the team value won't change because if I do remember well, the specifications Alexc made for teams was that users didn't take their points with them, much like boinc.
Those 8 million points were made when you were not on the team, so they won't belong to the team now that you join. But the points you make from now on will count towards the team total.

Anonymous
10-21-2005, 09:33 PM
Yes, I have discover that and I don´t like it. We should have over 1.000.000.000 url´s if we could get them to the stats totally. Today i got ca. 210.000 urls i think.
But moore will come when my A64 is back on Monday as they have said.

Lagu :)

Evil-Dragon
10-22-2005, 08:53 AM
The points/urls downloaded system for teams had to be done like this to stop unfair competition of one user with XX million urls leaving another team and taking all those XX million urls to another team.

It would cause a lot of problems, so it's best if it stays like it is.

Anyway at current crawling rates AMD Users team is doing very well.

Anonymous
10-22-2005, 01:34 PM
Yes Evil-Drogan, but there is a difference. If a people not is a member and join a team it should allow to add her/his stats to the totally. I have been a member of AMDusers in nearly one year. If I then join a another team it should be ok to let the ststs stay in the team.
However, we can have different options of that and I respect your comments.
We should get a great advantage if your ststs should be added to the totally. Surley impossibly for others to stomp us for long, long time forward.

Lagu :D

Lagu
10-22-2005, 01:37 PM
Damn, I forgot to log in twice, I must have got Alzeimers or is drunk or sleeping in front of the monitor.

Lagu :D :lol: :lol:

Evil-Dragon
10-23-2005, 08:24 AM
If a people not is a member and join a team it should allow to add her/his stats to the totally.

I'm not sure i totally understand, basically what i said above is the way that it is done to stop people being unfair to a team. I mean i've racked up 77 million urls since I started, i wouldn't really expect for my url contribution to be added to the AMD User team since it would create unfairness to every other new team and existing team.

Every team starts at 0 when it is created, any urls you do while in that team is credited to the team. If you leave the team then that work that you did will stay with the team. So basically if you join MJ12 and want your work units to be credited to a team then join it as soon as you begin crawling, that way there is no problem.

Ototero
10-23-2005, 09:35 AM
I agree Evil, all projects should be run that way.

I think Lagu's point was that he did not belong to a team so when he joins us (or someone else) those points follow him to the team. Then they become locked.

Lagu
10-23-2005, 03:47 PM
Yes Ototero, you are right. I thoug first I have added my points to the team but lather discowered my points not show up and was locked. This wasn´t what i have though but it play no role.
I may continue from start and try to work me up if possible.

Lagu :)

Lagu
10-29-2005, 11:09 PM
Hello

We have 14 partisipants in Majestic-12. I think two is new. We as is there are:

Vaughan (The leader), Australia
Beerknurd (The pinoneer), United States
NeoGen (The founder?), Portugal
Evil-Drogan, (Mr MJ-12), England
bwhite, United States
Lagu, Sweden
AMDave, Australia, (Not active)
anih, Poland, (New?)
outlawolf, United States
Chiron, Canada, (New?)
Ototero, United States
ahab3333, United States
arni, Tyskland
Empty_5oul, England

We as a team is number one in the world ranking list. Thank you all for all hours and money you all have used to get a such great results.
When was AMDusers number one last? never?

Lagu :D ;)

outlawolf
10-30-2005, 03:59 AM
i just hit a million URL's returned :)

Evil-Dragon
10-30-2005, 08:27 AM
Well done for hitting 1 million! On another note the project hit 2 bln urls crawled and another 500 mln (approx) will be loaded on Monday to crawl.

Hopefully we will at 3 bln crawled by christmas.

:)

vaughan
10-31-2005, 01:01 AM
I upgraded today and am running it as I have some spare bandwidth until the end of the month.
Question: How do you make it go full screen? It seems to be optimized for 640 x 480.

Anonymous
10-31-2005, 02:24 PM
Can't make it go full screen, it is a set dialog size.

vaughan
10-31-2005, 09:11 PM
Thanks Guest, its a pity as there seems to be lots of info to display.

Anonymous
10-31-2005, 09:32 PM
I beleive the Guest is Alexc himself because the message says: "I can´t make it..........."

Lagu :)

Lagu
10-31-2005, 09:34 PM
Damn, I forgot to log in.

Lagu :oops:

Lagu
11-01-2005, 07:24 PM
Hello

I’m running MJ-12 but have problems with the connection. I got to many DNSerror on my Athlon after running he node a while. This cause totally lost of the connection between the router and computer. If I tried to start Internet I can see how it tries the router address and then fail. I must reboot the computer and after that Internet works. However, if I run the node again the connection lost. If I run another project all works fine.
The router works fine because I don’t have any problems using the other computers.
I have checked Windows log and Telephony and WMI are started and I have tried to stop these but fail.

Any suggestion?

Lagu

ahab3333
11-01-2005, 08:34 PM
do you have the newest version?
it was just updated and i had the same problems

Lagu
11-01-2005, 09:10 PM
I apologice.

I was wrong. :oops:
It was not DNSerror, it was connection errors.

Yes I have but there is a lot of issues in the log.

Lagu :)

Lagu
11-01-2005, 09:52 PM
Hej

I have reset the router to factory settings because I think it was the routher as not want work properly. I may se if this solve the problems.

Lagu ;)