AMDave
12-31-2011, 09:28 PM
Renderfarm.fi gets ready for 2012!
We're happy to announce that we have now succesfully turned a page and finished the complete update of our server back-end. It was a rather gruelling task that took nearly two weeks to finish, but it brings us many of the improvements that we talked about with Jesse at this year's Blender Conference presentation:
Nathan explains in detail:
For some longer time we've been thinking of upgrading our back-end version of BURP to the latest. For our admins it brings some very important updates and improvements for managing the sessions that are running on the farm. We had scheduled to start the upgrade of the backend on December 20th, Janus had also promised to be available if any big issues would pop up. I had already prepared the development image we use with the latest BURP code and compiling it into a WAR-package was simple enough.
Jesse started the process by kicking off a Gentoo system update. Not very long into the update a fatal compile error got our hands all sweaty, as it told us that libc was completely broken. Fortunately a rerun of the update fixed this, but it also made sure that our adrenaline levels climbed high.
After initial deployment of the latest WAR-package it became obvious that some fixes on the farm were needed, especially where our frontend communicates with BURP. Fortunately a new XMLRPC API made the changes fairly easy, although on the other hand it was a break away from the old API we had in used (created last year by Olivier Romand). It took quite a while to get all the correct parts tied into each other. To be honest it was harder than initially expected. Jesse and Janus did most of the Apache/Tomcat configuration, while I continued working on the Uploader.
Then we ran into some mysterious problems. It turns out that the long type isn't officially supported in XMLRPC, and I had to spend some time on figuring out a working solution. The first quick fix provided by Janus worked, but not in all situations, and there were still places where long was being used.
I added some missing XMLRPC API to make sure the Uploader could be updated correctly too (the improvements will be committed to bf-extensions SVN soon). Just today the final few fixes (overlooked accidently yesterday) were made, ensuring that the service is again fully up and running! So, update of BURP included updating: Gentoo, Tomcat, Apache config files, fixing BURP XMLRPC code and the glue between our frontend and BURP.
Now that the update is finished and we're back to rendering, we can start focusing our efforts on making the new functionality as accessible as possible. We are planning to do this gradually by first testing their functionality with some actual production cases. Apart from this we'll have to also give consideration to how we will grant the right to use some of the features as they are potentially resource hogs (such as OpenEXR rendering, which can take tens of megabytes of storage per finished frame).
Apart from gaining some awesome new features, we can also put more effort into enabling Cycles support on Renderfarm.fi. This is turning out to be the single most requested feature in the farm's history. We hope to run preliminary tests in January, with at least limited support by February. Full implementation, as how we see it, would be a kind of a bucket rendering method which would enable amazing Cycles renders to be done in a completely distributed fashion. Let's cross our fingers the method we're putting our money on will work out!
Finally but not at all least importantly: We wish you all a Happy and Renderful New Year 2012! :-)The Renderfarm.fi posse for 2011: Julius, Jesse, Nathan, Janus, Eero & Davis
More... (http://www.renderfarm.fi/blog/prodigalson/renderfarmfi-ready-2012)
We're happy to announce that we have now succesfully turned a page and finished the complete update of our server back-end. It was a rather gruelling task that took nearly two weeks to finish, but it brings us many of the improvements that we talked about with Jesse at this year's Blender Conference presentation:
Nathan explains in detail:
For some longer time we've been thinking of upgrading our back-end version of BURP to the latest. For our admins it brings some very important updates and improvements for managing the sessions that are running on the farm. We had scheduled to start the upgrade of the backend on December 20th, Janus had also promised to be available if any big issues would pop up. I had already prepared the development image we use with the latest BURP code and compiling it into a WAR-package was simple enough.
Jesse started the process by kicking off a Gentoo system update. Not very long into the update a fatal compile error got our hands all sweaty, as it told us that libc was completely broken. Fortunately a rerun of the update fixed this, but it also made sure that our adrenaline levels climbed high.
After initial deployment of the latest WAR-package it became obvious that some fixes on the farm were needed, especially where our frontend communicates with BURP. Fortunately a new XMLRPC API made the changes fairly easy, although on the other hand it was a break away from the old API we had in used (created last year by Olivier Romand). It took quite a while to get all the correct parts tied into each other. To be honest it was harder than initially expected. Jesse and Janus did most of the Apache/Tomcat configuration, while I continued working on the Uploader.
Then we ran into some mysterious problems. It turns out that the long type isn't officially supported in XMLRPC, and I had to spend some time on figuring out a working solution. The first quick fix provided by Janus worked, but not in all situations, and there were still places where long was being used.
I added some missing XMLRPC API to make sure the Uploader could be updated correctly too (the improvements will be committed to bf-extensions SVN soon). Just today the final few fixes (overlooked accidently yesterday) were made, ensuring that the service is again fully up and running! So, update of BURP included updating: Gentoo, Tomcat, Apache config files, fixing BURP XMLRPC code and the glue between our frontend and BURP.
Now that the update is finished and we're back to rendering, we can start focusing our efforts on making the new functionality as accessible as possible. We are planning to do this gradually by first testing their functionality with some actual production cases. Apart from this we'll have to also give consideration to how we will grant the right to use some of the features as they are potentially resource hogs (such as OpenEXR rendering, which can take tens of megabytes of storage per finished frame).
Apart from gaining some awesome new features, we can also put more effort into enabling Cycles support on Renderfarm.fi. This is turning out to be the single most requested feature in the farm's history. We hope to run preliminary tests in January, with at least limited support by February. Full implementation, as how we see it, would be a kind of a bucket rendering method which would enable amazing Cycles renders to be done in a completely distributed fashion. Let's cross our fingers the method we're putting our money on will work out!
Finally but not at all least importantly: We wish you all a Happy and Renderful New Year 2012! :-)The Renderfarm.fi posse for 2011: Julius, Jesse, Nathan, Janus, Eero & Davis
More... (http://www.renderfarm.fi/blog/prodigalson/renderfarmfi-ready-2012)