Server Migration

This past weekend has been a busy one so far as Tailgrab’s back-end is concerned. The site was migrated off it’s “512” node to a “1024” as well as some architecture changes to how the site is served. The reason behind this move isn’t that the site required more resources, on the contrary, Tailgrab had more than sufficient resources on the smaller node. Rather, the move was to consolidate the resources I had between various nodes and simplify management.

The plan for this move had been in the works for a while, as I’d desired to move to Nginx as my web server in place of Apache2. The reasoning for this was an attempt to speed up the site as Nginx is known to be faster than Apache2 for static content. WordPress however isn’t static content and requires PHP for it to function, so what’s the point right? Well much of the content on Tailgrab is static content (images, js, css, etc), more over, I use a plugin for WordPress called WP-SuperCache which pre-caches all the pages on the site, both as html and gzipped html. This means that nearly the entire site can be served up as though it were static, skipping PHP for most content. There are however some remaining challenges to finishing off this transition. The use of timthumb to generate images in many places throughout the site requires PHP for every single hit. These images cannot be served statically via Nginx like normal images as Timthumb stores these images as text files in a caching directory of it’s own. Upon request for a timthumb image the request is passed to a PHP-FPM pool where it turns these text files back into the required image, each time requiring PHP to do so. Additionally, because these images have query strings in their URL, they’re not cached properly by browsers and proxies. Once I’ve removed this hurdle, the entire site should act as though it’s static and operate quite rapidly.

One of the biggest challenges in transitioning to Nginx from Apache2 is the lack of .htaccess files, or anything like them. What originally would have gone in an .htaccess file now must go directly into the sites configuration file, and Nginx restarted reloaded for them to take effect. Rather, the types of rules that would have gone in an .htaccess file, as Nginx uses a different rule structure they must all be translated to work. This becomes a big deal if you’d originally used .htaccess files to secure directories or if you’d turned on fancy URIs in WordPress or used WP-SuperCache. And since most plugins and PHP applications assume you’re using Apache, they might spew .htaccess files everywhere and you don’t want visitors to be able to access those. This means you need an additional rule to block access to such files, just in case they exist.

Overall it was a great learning process and I’d like to think I’m a better sysadmin now because of it. Hopefully everything continues to run smoothly, and just as importantly, hopefully I’ve managed to secure everything that needed securing. *fingers crossed*

-Aka

Comments

  1. Iwasawafag says:

    > and Nginx restarted for them to take effect.
    I think you know about it and phrase in above can be simplified for the reader, but I guess it’s will better to mention that restarting nginx isn’t necessary for most of changes and reload is usually enough for applying changes in config.
    And there is another one reason why using reload is prefer than restart. If you made some mistakes in config file, nginx will warn you about it, but your site(s) will keep working with previous loaded configuration.

    As usually, sorry for bad english

  2. The site seems quick and stable. Nice job with the migration! I personally don’t have a need for all the static-caching or even nginx. A fully optimized Apache with opcode caching PHP and additional memcaching at the application level is plenty fast enough for me but your setup seems to be working well for you ^^
    Did you also move Horobooru to the same server?

    • Aka says:

      Need? I didn’t need it either. But it seemed like an interesting project. I had a fairly optimized Apache with opcode caching PHP (APC) and WP-SuperCache there as well. I still have APC with Nginx, but Nginx can serve up the pre-gzipped site files from WP-SuperCache without having to compress on the fly or hit PHP.

      Horobooru was not moved, Tailgrab was moved to Horoboorus VPS. It already ran Nginx so it made some sense to just throw them together. Downside though is running two web frameworks and database applications. WordPress uses PHP+MySQL, Moebooru uses PostgreSQL+Ruby. The only commonality is Nginx in front. Would be nice to have them use similar backends, and I’m aware of the PHP port of Moebooru, but it’s not complete.

      • Iwasawafag says:

        You two are wizards. Last time when i tried to use APC and WPSuperCache together something went wrong and almost every request to the wp return code 500, empty page and no error messages.
        I found many exactly the same reports about “white screen of the death” with using apc and wp super cache plugin together on a php-fcgi, php-fpm and apache.

        • Iwasawafag says:

          Awww, this time I found a solution – adding a filter in apc section of php.ini. It seems more like a hot-fix to the problem, not a solving a reason of it. It’s not cool, but maybe i will try again.

        • Aka says:

          I use this plugin with WordPress and APC. Never had any white screens of death because of it. Other things for sure, but not that plugin specifically.

          As for Nginx and WP-SuperCache, I used this page for my rules. Though an older copy of it, as the current copy fails because of multiple “location /” rules. I then modified it to work with SSL.

          Why were you running php-fcgi and php-fpm at the same time?

          • Iwasawafag says:

            I didn’t running php-fcgi and php-fpm at the same time, it’s all because my english is shit, sorry. I meant all this guys in internet reported about this problem with any of that variants -fcgi, -fpm and mod_php. First time when I looked about this problem I suspected anything, even the environment.

            Thanks for the first link, I’ll try to use it. What about WP SuperCache — I use him in PHP-mode, so there is no need to have a special rules for wp super cache. I use it so because I wanted to make my customized comments much friendly for peoples with disabled javascript.