Joomla, DL() & You

I am not going to beat around the bush on this, the last couple of days have been a little hectic here a TCH while working to deal with a series of web application vulnerabilities that are being taken advantage of by attackers. The purpose of this post is to explain a bit about what is going on, how these attacks effect you and what we have done to prevent further abuse.

The first thing we need to understand is what is being attacked; as the post subject implies, it is primarily Joomla being attacked as the software has had a series of 9 vulnerabilities released since the 1st of September of which a number of more in depth attacks have formed around. The intended purpose of most of these attacks is to taint web sites with injected javascript, that code takes advantage of a number of client side browser vulnerabilities that if not patched or stopped by an antivirus can cause further issues for web site visitors.

Now, at a glance you might be thinking that if someone fails to patch web site software then it is there own problem, how does this affect me? That is where the dl() function comes into play, the dl() function is essentially a dynamic loader for php modules or 3rd party extensions. To simplify this a bit, the dl() function when enabled allows anyone to add extensible features onto php, generally these are all well to do features but if someone so desires they can create a dynamic loader module with malicious intent.

The scenario we are looking at is that attackers have gained entry to vulnerable web sites, primarily through joomla then they upload a series of malicious scripts including a dynamic loadable module for php that once enabled through dl() has the ability to inject javascript code into pages. The code usually finds itself placed before the body tags and executes its payload on a visitors first visit to a site, a cookie is then set that expires every 2hours then the payload executes itself again on a new visit.

This attack though had far reaching implications, only affected 4 servers on our network (denver, dantooine, alderaan, chewbacca) of which only about half the sites on the given servers or in some cases less were being tainted by the attack. As alarming as this situation is, we need to stress that no content was actually modified on sites except the joomla sites themselves that were compromised.

The way we have come to deal with this situation is a layered approach, we have first and foremost made increased efforts to identify compromised sites on our servers and suspend/remove them. The next step was to cut off the enabling function of the attack, which is the dl() function. This function was actually something we used to disable on servers for its malicious implications but over time that procedure was phased out in the interest of allowing users to install custom dynamic loadable modules from their home directories such as ioncube. However, now that ioncube is standard server-wide on all servers, there is little in the way of other commonly installed packages that depend on dl(), has even went as far as to declare dl() deprecated as of php 5.3.

With dl() disabled on servers, the effects were immediate and all reports of tainted sites stopped, now when I say stopped I do not just mean that that lightly. We literally sat around all evening bashing the f5 key on our keyboards trying to get the javacode injections to reappear on sites, between myself, Bill and Dick we must have done over 6 hours of combined keyboard kungfu in this effort. It was with great relief that we were not seeing anymore reports or issues ourselves first hand but it was still not quite enough to actually be confident that we had done enough.

We are continuing to be extra vigilant with compromise assessment on the servers to prevent any further malicious content from being injected into sites, in addition to this we have on some servers started to use suPHP as a basis for new php security standards. Essentially, by using suPHP we enforce php code to run as the user who executed it instead of as the web server but it goes beyond that by enforcing strict permissions on content and not allowing anything to run above mode 755 (such as world writable data) and also making sure that executed content is owned by the user. This might seem problematic however since the code is now executing as the user, there is no longer a need for data to be set to mode 777 (world writable) or its ownership set as the web server user, which reduces support issues and vastly increases security. The suPHP changes are something we have only rolled out to about 6 servers so far but the support issues it has generated are minimal for the advantages it provides, in the future we will be looking to roll this change out to more servers on a slow but steady basis.

That is where we are at, if you have any questions or concerns regarding this blog or the topics discussed please feel free to comment or head to the TCH forums for further dialog.


2 thoughts on “Joomla, DL() & You

  1. I would like to take the time to thank all of you for taking your time to sit there bashing the F5 key. I spent hours myself debugging it on a few of my sites, including trying to record screen videos of it happening which in itself took over 2 hours, and reading my firebug console outputs and scanning all my personal files.

    I am unreservedly grateful for your guys’ hard work! I doubt there would be other hosts who would take that time out to source and fix this sort of issue.

  2. I’ve just discovered the blog, and let me say I found it really interesting to read what is going on behind the scenes. I wasn’t troubled by the Joomla issue, but have experienced several other site cracking attempts, and am also on one of the servers that went to suPHP – while that caused a few teething troubles, I can see the benefit in it.
    It really made me feel part of the TC community again, reading over some of the previous posts (have been too busy to get onto the forums much, I’m sorry to say)

    Thanks and keep up the blog posts.