Page 15 of 25 FirstFirst ... 56789101112131415161718192021222324 ... LastLast
Results 141 to 150 of 246

Thread: bok's stats

  1. #141
    Join Date
    Nov 2005
    Location
    Central Pennsylvania
    Posts
    4,333
    Raid explained the best methodology I can find out there: http://www.thegeekstuff.com/2010/08/...evels-tutorial & http://www.pcmag.com/article2/0,2817,2370235,00.asp

    In the numerous years I have had my system and all its memory moved from system to system, my tech has never lost anything for me. Yes I had a drive completely fail, but I had my system in a mirror setup and the other drive was fully operational and all assets were recovered. Yes to support Vaughan's statement we only crunching but some of us have other material on our systems which require retrieval if it ever fails.





    Challenge me, or correct me, but don't ask me to die quietly.

    …Pursuit is always hard, capturing is really not the focus, it’s the hunt ...

  2. #142
    Join Date
    Sep 2010
    Location
    Leiden, the Netherlands
    Posts
    4,372
    Much too basic!
    RAID 0 (Shit, but fast. Suicide for dummies and gamers. One drive gone, everything gone. Don't try this at home. The more disks you use in RAID 0, the faster it gets -reading and writing) .
    RAID 1 (Mirroring only, does not enhance performance one bit and it is a lousy way of backing up. The more disks you use in RAID 1, the slower it gets).
    RAID 5 (RAID for cheap skates: it will allow one disk failure, but all-to-often a second disk fails just when replacing the first. The more disks you use in RAID 5, the better -faster in reading- it gets).
    RAID 6 (RAID for less cheap skates: it will allow two disks failures, but all-to-often a third also disk fails just when replacing the first two. The more disks you use in RAID 6, the better it gets).

    You can also try to combine raid levels.
    RAID 10 (= RAID 1 and RAID 0 combined= Mirrored Shit).
    Having two -or more- RAID 5 arrays running under RAID 0 gets you RAID 50.
    But you should go, IMHO, for RAID 60 which is also known as dual drive failure protection and is built from -at least- eight disk drives configured as two -or more- RAID 6 arrays, and stripes stored data and two sets of parity data across all disk drives in both -or more- RAID 6 arrays.

    Performance-wise, why not use 8 arrays of 8 disks each? Okay, that's 64 disks, but each 8-disk array may lose up to two disks before it gets critical.
    Last edited by Dirk Broer; 10-08-2016 at 12:04 AM.


  3. #143
    Join Date
    Jul 2003
    Location
    Sydney, Australia
    Posts
    5,642
    RAID 60 sounds expensive. Why not use a Cloud backup company such as Backblaze, Crashplan, Spideroak or Carbonite? There are others but I haven't tried them.

  4. #144
    Join Date
    Sep 2010
    Location
    Leiden, the Netherlands
    Posts
    4,372
    Backup should never be a purpose of RAID, not even when using RAID 1.
    RAID should be used to ensure uptime, read- and/or write speeds.
    RAID 60 can already be done with two minimal RAID 6 arrays (4 disks each) running under RAID 0.
    A site as FreeDC, especially the stats pages, could benefit from a RAID 60 setup.


  5. #145
    NeoGen's Avatar
    NeoGen is offline AMD Users Alchemist Moderator
    Site Admin
    Join Date
    Oct 2003
    Location
    North Little Rock, AR (USA)
    Posts
    8,451
    Make it a RAID 60 of 1Tb SSDs with a powerful RAID adapter and you got yourself one heck of an I/O workhorse.

    8x 1Tb standard SSDs drives (2.5'' form factor) will cost over $2000 minimum... it's really expensive but the responsiveness and read/write data bandwidth would be amazing! Something that could only be surpassed by dedicated storage cluster servers or NVMe dedicated arrays.
    Last edited by NeoGen; 10-11-2016 at 12:05 AM.

  6. #146
    Join Date
    Sep 2010
    Location
    Leiden, the Netherlands
    Posts
    4,372
    8x 1 TB or
    16x 512 GB or
    32x 256 GB, why not
    64x 120 GB drives?


  7. #147
    AMDave's Avatar
    AMDave is offline Seeker of the exit clause Moderator
    Site Admin
    Join Date
    Jun 2004
    Location
    Deep in a while loop
    Posts
    9,609
    Sounds to me like your aspirations may be greater than your needs ;)

    To dig up an old quote that directly lends to an old principle that I was taught a very long time ago and I still think is fun:
    "Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway." (Computer Networks, 4th ed., p. 91., Andrew S. Tanenbaum)
    https://en.wikiquote.org/wiki/Andrew_S._Tanenbaum

    Very little we do as individuals outside of the commercial environment is important enough to warrant "live" on-site and off-site mirror backup.
    Some consideration around 2 metrics will illustrate this for most of us with a grasp of risk analysis and Disaster recovery: RTO and RPO
    https://en.wikipedia.org/wiki/Recovery_point_objective
    https://en.wikipedia.org/wiki/Recovery_time_objective

    Once you figured that out you should realize that 1 extra large internal HDD inside a workstation / server + 1 large external NAS (on-site) + 1 large external NAS (offsite) handles your issue for a fraction of the price.
    The large HDD keeps the immediate backups available.
    The 2 externals NASs are rotated to keep a regularly updated copy of the internal HDD; frequently refreshed, portable and in a risk-reduced location
    (ie the on-site NAS is just as screwed as the internal HDD if your house or office falls into a sink hole or gets burned down by a neighbor kid's charming little experiment with matches or flattened by the cyclone tearing across the coastline)

    If you want fast AND reliable then split the 2 functions.
    RAID-0 SSD for primary disk is awesome. No argument.
    Local HDD(s) for backup is very useful esp for RTO and RPO. No argument.
    It is easily enhanced by on and offsite external NAS backups in rotation for assurance.

    A local NAS serves as a part of the grab-and-go kit (along with a charged laptop) in case of emergency evacuation.
    A remote NAS serves as a recovery point if the local site is destroyed.
    In the mean time the local backup HDD(s) give you as much recovery as you can fit (add more disks as required)
    Occasionally swap the local and off-site NAS copies to keep the RPO low.

    A lot of NAS devices have RAID options and mirror should be used for protection when it is available.
    When a 1TB or 3TB disk fails be prepared for a very long image catch-up when you plug in the replacement.
    That process can take a day or 2 going on current experience.
    But remember that is happening on your backup set not in your live environment - so minimal interruption unless it happens while you are trying to recover in the case of a real DR.

    IMHO Easy. Fast. Cheap. Reliable. Private. Secure. No 3rd party reliance, contracts, access restrictions, costs.

    As far as the myriad RAID configurations available - "Just because you can doesn't mean you should" ("Lest We Forget", Lt. Col. Carlos A. Keasler)

    re: SSD mirror RAID Given the design limitations of SSDs, putting a new SSD into a mirrored SSD RAID configuration to replace another that has failed will shorten the life of the new SSD through the huge number of writes that will be processed to catch it up
    (although it is still faster than catching up a HDD you are compromising speed for a false sense of reliability)

    re: NVMe is fast. NVMe is coming down in price, but currently still at approx. 4 x $/GB compared to SSD.
    We will probably see it half and maybe even match price in the next year.
    But the rational outlined above still stands whether NVMe or SSD is considered.
    ITMT still in the server-level realm unless you have money to burn - in which case, could you buy me one too? for testing of course. I'm sure I'll get it back to you oh sometime ;)
    Like SSDs - still best used for high read rates and very low write rates.

    I'd hate to miss this opportunity to remind everyone:
    Your last backup is only as good as your last restore.
    An untested backup is NOT a reliable backup.
    Having observed organisations large and small and small business owners and individuals suffer from failed restores from backups, I have always tested my backups right after I make them.
    Some of the early ones did not work well and highlighted issues and false-faith in common backup software and led me to improve my own backup and restore procedures to the point where I have not had a restore fail for many years now.
    Please do not waste any time doing backups that you don't know will work when you suddenly really need them. That's just pointless.
    When you blast the summer dust bunnies out of your PC this Autumn, do yourself a favor and make a fresh backup set and store it in a dry box at a different address.
    You cannot go back in time and re-take all those family photos, re-scan the tax invoices you threw out last year. (well not yet anyway. maybe it will happen in our lifetime. maybe not. how confident are you?)

    ed - coincidentally, our newegg partner is running a clearance on storage devices. clik on the newegg promo logo at the top of the forum. or > here <
    Last edited by AMDave; 10-11-2016 at 12:00 PM. Reason: tpyo & promo
    . . . . . ___
    . . . . . . .\___/\______
    . . . . . . . \__AMD___\\__
    ---------------------------------------------

  8. #148
    Join Date
    Sep 2010
    Location
    Leiden, the Netherlands
    Posts
    4,372
    I was purely thinking in FreeDC's problems. Bok needs:
    1. uptime
    2. fast reads
    3. fast writes
    4. to be able to change a failed disk without having to take the site down

    He would be served by a 64-disk RAID 60 solution, IMHO.


  9. #149
    AMDave's Avatar
    AMDave is offline Seeker of the exit clause Moderator
    Site Admin
    Join Date
    Jun 2004
    Location
    Deep in a while loop
    Posts
    9,609
    Sure. As long as you are talking about HDDs.

    Mirror RAID for consumer SSDs is also a really over-confident idea and can result in complete loss when they start failing together.
    eg: It looks like I'll soon be re-building a $16K SSD raid array on a business machine. Fortunately I saw an indicator their SSD mirror RAID was going bad - so I installed a 3TB internal disk and built in a backup process, just like I outlined above. I am anticipating total failure of the RAID volume sooner than we can replace all of the drives in the array.

    SSD mirrror RAID is unreliable over the long term.
    If one goes bad you need to be replacing the lot of them.
    In which case, why mirror raid SSDs?? Experience shows it to be a waste of time and money relative to the write-rate.
    You want them for speed, so RAID-0 & do some backups to another more reliable media.

    For Bok, there is an additional couple of lines of script that could resolve the current symptom & allow SSD replacement to occur without downtime in the current architecture by detecting the issue and pausing the web-side DB flip-flop.
    It's something I thought of since we discussed the implementation of the current architecture, but thought he would never need.
    Little did we know just how crap these consumer SSDs would actually turn out to be for this kind of usage.

    Now if there was tons of RAM available ... maybe copy it to RAM disk and back that up to HDD and skip the SSD altogether.
    Last edited by AMDave; 10-11-2016 at 07:45 PM.
    . . . . . ___
    . . . . . . .\___/\______
    . . . . . . . \__AMD___\\__
    ---------------------------------------------

  10. #150
    Join Date
    Sep 2010
    Location
    Leiden, the Netherlands
    Posts
    4,372
    We could try to open a new instance of Bitcoin Utopia's campaign #12, for 32 GB registered ECC (no doubt) RAM modules in Bok's box (if his box takes such biggies).


Page 15 of 25 FirstFirst ... 56789101112131415161718192021222324 ... LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •