Cygwin Lighttpd with SSL
Last week, I needed to configure a Windows 2003 AMI in EC2 to run lighttpd with SSL. Once again, a simple job that I thought would be quick and painless turned into an extensive hacking session. Here is a quick roadmap of what I did.
My initial thought was that there must be a native port of lighttpd with SSL support for Windows…wrong again! I don’t know why I continue to think that open source projects will be ported to Windows; Linux is long since mature enough that people can use it directly, rather than porting open source Linux software to Windows.
In any event, cygwin lists lighttpd in its package list, so I was left with the option to run lighttpd via cygwin. A few quick clicks of the cygwin installer gave me lighttpd 1.4.20-1, all installed and ready to start. I dropped in a lighttpd.conf file containing ssl.engine declarations and started the server. Lighttpd promptly informed me that SSL support had not been compiled in.
I really have no idea why lighttpd on cygwin ships without SSL support compiled in, especially since the openssl libraries are available within cygwin and SSL support can be compiled in with just a simple option to ./configure. But, I should just be able to download the lighttpd source and compile it myself, on cygwin, with SSL support enabled, right?
Wrong again! After downloading lighttpd source for 1.4.20 and installing gcc on cygwin, I ran ./configure with the following options:
./configure --disable-shared --enable-static --prefix=/usr --with-openssl
I kicked off the build with make, and went to grab a beer (the build proceeds very slowly). When I returned, I found the build had failed with a symbol undeclared error for EAI_SYSTEM. This error has lots of search hits and some hacking-oriented solutions, but none of those solutions seemed to work for me, owing to conflicts between the static build of lighttpd and the main cygwin DLL. Even when I did manage to get the build to successfully complete, I found that some features (e.g. CGI execution) just didn’t work. The last thing I want is a flaky build of lighttpd running within cygwin on Windows, so I needed a plan B.
After a few more searches, I came across the WLMP project. WLMP has a standalone build of lighttpd — optionally bundled with MySQL and Php — that includes the cygwin DLLs but doesn’t otherwise require or run within cygwin. Funky, but promising. I installed the lighttpd-only version of WLMP, and I was able to bring lighttpd up with support for SSL, but only as a stand alone application, outside of cygwin. If I tried to run it from within a cygwin shell, or even with the cygwin DLL loaded, I found all sorts of silent errors without any messages. I intended to use cygwin extensively while lighttpd was running, so this was a significant problem.
On to plan C. I’m starting to think this isn’t going to happen when it occurs to me that the WLMP lighttpd build is a cygwin build with some conflicting cygwin files. If I download a version of WLMP that uses the same lighttpd 1.4.20 version as cygwin, and remove the files that conflict, then maybe I can get the WLMP build to run in cygwin. It’s worth a shot, right?
After some trial and error, I was able to make this work as follows:
0) Download WLMP lighttpd 1.4.20, matching the version of lighttpd in cygwin.
1) Delete the Cygwin1.dll file from the WLMP lighttpd directory
2) Add execute permission to all of the DLLs in the WLMP lighttpd directory, and all of the mod DLLs in the WLMP lighttpd/lib directory.
3) Cd into the WLMP lighttpd directory, and from within the WLMP lighttpd directory start lighttpd with the following command:
./lighttpd -V -m /usr/lib/lighttpd
If all went according to plan, you should see the list of features compiled into lighttpd, and there should be a “+” sign in front of “SSL Support”. My next test was to start lighttpd with the same –m argument, and using my lighttpd.conf with the SSL configuration, and it worked flawlessly without any cygwin conflicts.
I can’t tell you why the current directory (within the WLMP lighttpd directory) is important in getting this hack to work. My guess is that the WLMP version of cygwin is using a combination of DLLs from the standard cygwin lighttpd lib directory in /usr/lib/lighttpd and the WLMP lighttpd lib directory in the current directory, and somehow this combination works without cygwin DLL conflicts in this particular situation. Who knows? All I can say is that it would have been really nice if the cygwin folks had enabled SSL support in their build of lighttpd…
Anyway, that’s it. Hope that helps.
Security for Cloud-based Enterprise Applications
Being the architect of a cloud-based enterprise software-as-a-service (SaaS) product — Adobe LiveCycle Express — I get asked a lot about the security of cloud-based applications. They are all the questions you would expect: “Is my data secure in the cloud?”; “How safe are cloud service environments?; “Aren’t you just asking to be hacked?”. My response to all of these questions is, first, to tell them to take a deep breath and relax, and second, to think of the cloud as any other computer system faced with security threats, and for which you need to develop a threat response. Security for cloud-based applications is actually a multifaceted problem, with distinct threats and responses for network security, operating system security, data security, and virtualization security — but each of these distinct areas has an analog in non-cloud-based applications that has been analyzed and for which there are security procedures to help reduce threats. So, ultimately, we just need to ignore the hype surrounding cloud computing and focus on the fundamental computer system security issues that applications will face in this domain. Some brief commentary on each of the threat areas faced by cloud-based applications follows, along with a note about regulatory issues. As with anything you read on the internet, your mileage may vary.
Network Security for Cloud-based Enterprise Applications
The ideal network security paradigm for cloud-based enterprise applications would be to have the cloud services be an extension of the customer’s existing internal networks, using all of the same protection measures and security precautions that their administrators have developed. This implies a strategy that would allow a customer to extend their network security envelope to encompass the cloud services that they use. For my application, I have chosen to use an encrypted TCP port forwarding strategy to extend the customer’s network security envelope into the cloud. Specifically, I use an implementation of the ssh protocol that performs bi-directional TCP traffic forwarding across encrypted, compressed connections between cloud-based virtual instances and customer machines behind their corporate firewalls. The encryption strength is configurable, and is at least as secure as any corporate VPN encryption. This is another way of saying that it does not introduce any new network threats beyond what a customer already faces with their existing VPN technologies. Less paranoid customers can choose to downgrade the security of their cloud instances and access them directly via HTTP and HTTPS, but my recommended approach is to use encrypted traffic forwarding, both for robust security and to translate the problem of cloud-based network security into terms and concepts — VPNs and encryption — that system administrators will understand. In reducing the problem of cloud-based network security to one of corporate network security, we simplify the problem of assessing and responding to cloud-based network security threats.
Operating System Security for Cloud-based Enterprise Applications
Cloud-based virtual instances face the same OS-level threats as standard hardware-based installs. The typical responses for these threats are the same as for hardware-based installs (e.g. block all non-essential listen ports via firewalls, keep the OS patches current, and run modern anti-virus software). For my enterprise application, I recommend that all ports except the ssh port (port 22) are to be blocked by firewall, patches are to be updated periodically and rolled into new virtual images, and antivirus software is to be installed; these measures collectively ensure that the virtual instances face no new OS-level threats beyond what a customer already faces with their existing hardware. As with network security, by reducing the problem of cloud-based operating system security to one of traditional hardware-based operating system security we simplify the problem of assessing and responding to threats.
Data Security for Cloud-based Enterprise Applications
In cloud-based applications, data encryption is the simplest way to protect data. Data in a cloud-based application will likely be transiting public networks and shared resources, and it makes sense to protect it against simple observation. Modern encryption technologies will withstand all but the most determined attempts to crack them, and while they require some key sharing infrastructure to implement them properly, they can be deployed using traditional methods employed for non-cloud-based applications. There are potential performance implications for data encryption, but, in my opinion, when using modern hardware and fast networks with a well-designed application, the performance implications can be effectively minimized. In my application, virtual instances store their backups off-instance within a cloud storage service. These backups are encrypted using a PKI key pair, with only the public key stored on the virtual instance. The data storage needs of virtual instances in my application are simple and infrequent, so the application easily lends itself to a data encryption strategy. Not all applications will have these characteristics; however, there are very few alternatives that can ensure data security in cloud-based applications. Thus, data encryption in some form will likely be a necessary evil of all cloud-based applications…but it’s not really that evil. And once again, we have reduced the problem of data security in cloud-based applications to an existing, non-cloud-based technology domain.
Virtualization Security for Cloud-based Enterprise Applications
Cloud-based virtual instances face resource-sharing threats that are inherent to the virtualization environment. These threats, however, are similar to the resource-sharing threats seen by enterprise applications that share hardware resources, either by design or necessity, within corporate data centers, and for which there are some procedures and assumptions that can be put to use in the cloud domain. For cloud-based resource-sharing threats, the problem is largely within the domain of the cloud virtualization provider, and it is their responsibility to respond to the threat. Customers of a virtualization service are ultimately trusting the virtualization service provider to guarantee the inter-instance security of the virtual instances, much as shared applications on common hardware are trusting the operating system to enforce inter-process security. Cloud-based applications rely on cloud services to enforce inter-instance security and guarantee that shared memory and disk resources can not be used to access another customer’s applications. Many cloud service providers run their own business applications within the same virtualization fabric, so it is fair to assume that inter-instance security is being monitored by corporate security professionals. In my opinion, the security budgets of the major cloud service providers far exceeds the security budgets of most cloud service consumers, and thus the degree of virtualization security provided in these service environments will be at least as good as the resource-sharing security found within a customer’s data center. For example, Amazon runs their flagship business within the same data centers as their cloud services, and I imagine that they have some of the best security professionals in the industry ensuring that their business applications are not at risk to resource-sharing threats from their cloud services. Ultimately, however, the threat is not substantially different than the hardware-based shared resource threats that customers currently face in their own datacenters. We can reduce the issue of virtualization security to one that is similar to an existing threat, and for which there are standard practices to assess and respond to that threat.
Regulatory Issues for Cloud-based Enterprise Applications
A discussion of security for cloud-based enterprise applications would not be complete without some comments on the regulatory issues that come into play when enterprise applications, and specifically enterprise application data, are moved into cloud-based services. Legislation and regulation in many industries has led to strict guidelines regarding enterprise data, primarily to protect the privacy of individuals. For example, The Health Insurance Portability and Accountability Act (HIPAA) contains strict provisions on what can be done with individuals’ health records, and what permissions must be explicitly received from those individuals for any transfer of those records. For an enterprise application that contains health records, this presents an open question that must be answered before that application can integrate cloud services into its architecture: How do the regulations within HIPAA apply to cloud-based services, and what restrictions do they place, if any, upon the use of third-party cloud services within enterprise healthcare applications? The answer to this question, and similar questions for regulations in other industries, is evolving rapidly, but it will ultimately be answered by legislation and regulation rather than by technology. However, progress towards an answer will be made by software vendors and enterprise customers that make the case that threats faced by cloud-based enterprise applications are ultimately no different, and no more severe, than the existing threats faced by those applications in their current deployment formats.
Summary
I have tried to make the argument that the security threats seen by cloud-based enterprise applications are translations of threats seen by existing enterprise applications within corporate data centers, and thus there are existing procedures and responses that can be put to use in designing an appropriate security strategy. To paraphrase Bruce Schneier, security is theater, and the perception of security for cloud-based enterprise applications is likely more important than the actual manifestation of the security mechanisms. This has been true at each evolutionary stage for enterprise systems — does everyone remember the paranoia surrounding the first attempts to connect corporate networks to the public internet? — and the continuing deployment of ever more cloud-based applications will need to reach a critical mass, with respect to the perceived security story, before it becomes effectively mainstream. But that day is coming.
Hope this helps.
Cygwin SSHd on a Windows 2003 AMI Within Amazon EC2
Recently, I needed to configure a Windows 2003 AMI in EC2 to run a ssh server. I would have expected this to be a simple job, with a variety of choices for making this work, but in the end it was far more time consuming, complicated, and frustrating than I would have guessed. Here is a quick road map of what I did.
My initial thought was that there must be a free, native port of openssh for Windows that installs as a service and otherwise conforms to the Windows environment…wrong! I can’t tell you why this is the case — maybe ssh is just not a microsofty way of doing remote terminals and file transfers — but I couldn’t find anything resembling a free, functional port of openssh for Windows. I found a few blog posts that mentioned that people had tried this, but ultimately they gave up when faced with the integration between openssh’s user/group namespace functions and Windows’ user/group concepts (to say nothing of the differences between the Windows command prompt and the UNIX shells). And these blog posts ultimately suggested that it was easier to run sshd via cygwin than it would be to port sshd to run natively. So….cygwin time!
UNIX is my OS of choice, and I’ve had cygwin on every Windows box I have ever had, so it was a quick jump to download the cygwin installer and install the packages I needed on a freshly started Windows 2003 instance in EC2 (incidentally, I am running the 64-bit, large EC2 instance AMI of Windows 2003 Server with SQL Server Express and no Authentication Services). The openssh package comes with a simple script — ssh-host-config — to generate the server host keys and create the users needed for privilege separation, so it was a nice, simple, relatively painless install. There are a few things that the config script misses, however, which requires you to run it several times before it ultimately succeeds (although it is nice enough to point out the problem each time and prompt you to fix it). After playing with it, I came up with the following actions to perform before running ssh-host-config in order to make it succeed the first time without errors:
0) Add the following line to /cygwin.bat:
set CYGWIN=binmode tty ntsec
1) Run a new cygwin bash shell (after the edit of cygwin.bat) and enter:
mount -s --change-cygdrive-prefix /
chmod +r /etc/passwd /etc/group
chmod 755 /var
2) Run a new cygwin bash shell (to pick up the cygdrive prefix change) and enter:
ssh-host-config
-- yes for privilege separation
-- "binmode tty ntsec" for CYGWIN environment variable setting for the service
-- enter your password of choice for the cyg_server account
3) Enter the following to start sshd:
net start sshd
4) Open the Windows Firewall editor, and add an exception for TCP traffic on port 22 for sshd.
5) If you haven’t already done so, open up port 22 for your EC2 instance group (assuming you are running your instance in the default group):
ec2-authorize -p 22 default
If everything went well, sshd is running and available on port 22, and you can login normally via ssh from other machines. All that is left to do is bundle up a new AMI to capture the cygwin installation…and that should be a piece of cake, right? The updated EC2 API has a new method — ec2-bundle-instance — that kicks off an AMI bundling job for an EC2 instance running Windows, so it should be as simple as calling this method and then grabbing a beer to wait for it to complete. If only it were that simple…
Unlike the AMI bundling scripts for Linux-based EC2 instances, which are ultimately just packaging up the existing file system, the Windows AMI bundling mechanism needs to perform several Windows-specific functions that are ultimately a real pain in the neck. First and foremost is sysprep. Sysprep is Microsoft’s answer to the problem of Windows virtualization; apparently the simple cloning of a Windows installation is not acceptable, and a new Windows SID should be generated for each new instantiation of a Windows virtual image. Sysprep does some other things, too (search for sysprep on Microsoft’s support web site for a more complete description — I am certainly not an expert on it), but ultimately the SID generation is the one that causes problems for a lot of installed software…like cygwin. After bundling a new AMI and starting a new instance with it, I found that sshd is hosed for no apparent reason. Attempts to start sshd via “net start sshd” produce the following cryptic error message:
The CYGWIN sshd service is starting.
The CYGWIN sshd service could not be started.
The service did not report an error.More help is available by typing NET HELPMSG 3534.
WTF?
After several time-consuming iterations of start new instance -> install cygwin -> bundle new AMI -> start new AMI instance -> wonder why sshd is hosed, I found something in the HKEY_USERS tree of the Windows registry that changes after the bundling step. Prior to bundling, with a functioning cygwin/sshd, I see the following in the registry:
[HKEY_USERS\S-1-5-21-2574196159-1727499900-3384088469-1013\Software\Cygnus Solutions]
[HKEY_USERS\S-1-5-21-2574196159-1727499900-3384088469-1013\Software\Cygnus Solutions\Cygwin]
[HKEY_USERS\S-1-5-21-2574196159-1727499900-3384088469-1013\Software\Cygnus Solutions\Cygwin\mounts v2]
[HKEY_USERS\S-1-5-21-2574196159-1727499900-3384088469-1013\Software\Cygnus Solutions\Cygwin\Program Options]
[HKEY_USERS\S-1-5-21-2574196159-1727499900-3384088469-500\Software\Cygnus Solutions]
[HKEY_USERS\S-1-5-21-2574196159-1727499900-3384088469-500\Software\Cygnus Solutions\Cygwin]
[HKEY_USERS\S-1-5-21-2574196159-1727499900-3384088469-500\Software\Cygnus Solutions\Cygwin\mounts v2]
[HKEY_USERS\S-1-5-21-2574196159-1727499900-3384088469-500\Software\Cygnus Solutions\Cygwin\Program Options]
After bundling, in a new instance in which sshd is hosed, I see the following in the registry:
[HKEY_USERS\S-1-5-21-4261372910-2505678249-1238160980-500\Software\Cygnus Solutions][HKEY_USERS\S-1-5-21-4261372910-2505678249-1238160980-500\Software\Cygnus Solutions\Cygwin]
[HKEY_USERS\S-1-5-21-4261372910-2505678249-1238160980-500\Software\Cygnus Solutions\Cygwin\mounts v2]
[HKEY_USERS\S-1-5-21-4261372910-2505678249-1238160980-500\Software\Cygnus Solutions\Cygwin\Program Options]
All of the other registry entries related to cygwin remain the same before and after the bundling step, so my guess is that the loss of entries in the bundled instance is the source of the trouble. But what exactly are those entries?
Again, I’m no windows expert, but the entries in question appear to have the windows SID followed by a user identifier (e.g. in S-1-5-21-4261372910-2505678249-1238160980-500, S-1-5-21-4261372910-2505678249-1238160980 is the SID, and 500 is the user id). Looking at the /etc/passwd file for cygwin, the user id 500 corresponds to the Administrator account, and user id 1013 corresponds to the cyg_server account, used by sshd as a privileged account for switching effective user ids during login. So, my hypothesis is that the privileges for the cyg_server account are somehow lost by sysprep during the bundling step, and sshd is hosed without them in the new bundled AMI instance.To test my hypothesis, I decided to configure the AMI bundling step to skip sysprep. The base Windows EC2 AMIs come with an application in the start menu called “ec2Service Setting” that has a check box to enable/disable sysprep during AMI bundling, so it is easy enough to test this. However, I have no idea what happens to Windows if I disable sysprep during bundling, and I was not able to find a satisfactory answer via internet searches. The closest I got to an answer was to see several of the Amazon admins on the EC2 forum comment that it was not a good idea to disable sysprep if you were going to instantate multiple instances. I also found several documents online that discussed how sysprep was used to sanitize a Windows installation, generate a new SID, and make it generic for installation on any type of hardware. Since the virtual hardware of EC2 is, roughly speaking, identical (given that it is using Xen underneath the hood), I’m not too worried about the hardware issue. I have no idea about “sanitizing” the Windows instance or SID generation, though, so bundling without sysprep might mortally wound Windows (again…I’m no Windows expert). And I do want to run multiple instances from the bundled AMI, so that might be a non-starter as well. So I guess I will try the ready-shoot-aim approach of seeing what happens when I turn it off…
Compressing time, I started with a fresh Windows instance, installed cygwin and configured sshd like before, turned off sysprep and bundled it, started a new instance from the new bundled AMI, and…sshd still works. The new instance retains the SID that it had prior to bundling, and the registry entries are still there for the cyg_server account. Windows also appears to be working in all respects, but I’m not sure I could detect problems that might result internally from the omission of sysprep in the bundling. I guess I can run one more test, starting a bunch of instances at once, to see if having the same SID causes them to interfere with one another. I started four instances, running concurrently, and they each seem to be working fine. Or at least I can’t detect any problems.
So, in closing, it looks like I may have a solution: turn off sysprep if you want to use cygwin sshd in a bundled Windows AMI. Someone with more Microsoft kung-fu might be able to figure out how to make sysprep retain the registry entries for the cyg_server account, or maybe they would write a script to insert them directly into the registry at restart if they are missing…who knows. But for me, disabling sysprep seems to be the way to go. I found lots of other complaints on the internet about sysprep and what it does to installed software when the SID changes, so I’m guessing that there will be a lot of bundled AMIs in EC2 that are created with sysprep disabled. If there are, in fact, issues with multiple instances using the same SID, then I expect we will be reading about it in the EC2 forums, since everyone who creates a new AMI from the base Windows AMIs without sysprep will have the same base SID in their AMIs, and so on….
Anyway, that’s it. Hope that helps.
My experimental search engine is now available in 10 languages…
About 18 months ago, I created an experimental search engine to play with some new ways to extract and distill information from the web and present it in a more topic-focused way. The site is called doryoku (which is Japanese — 努力 — meaning supreme effort). It has been doing well, and my experiments and enhancements continue. My latest addition involves automatic translation: the site is now available in 10 languages: English, Japanese, Chinese, Korean, French, German, Italian, Spanish, Russian, and Greek. The translation capabilities are courtesy of the Google Language API.
More new features to come. Stay tuned!
Adobe LiveCycle Express — Cloud Computing Meets Adobe LiveCycle ES
Adobe LiveCycle Express is a new software-as-a-service (SaaS) product offering from Adobe that takes a cloud computing approach to delivering Adobe’s LiveCycle ES enterprise software suite using Ruby on Rails, Adobe Flex, and Amazon’s EC2 and S3 services. LiveCycle Express grew from a research project I conceived while working in Adobe’s Advanced Tchnology Labs division.
The site was launched on December 15, 2008 at http://livecycle.express.adobe.com.
The joint press release from Adobe and Amazon announcing LiveCycle Express can be found here: http://finance.yahoo.com/news/Adobe-Announces-LiveCycle-bw-14025946.html.
Installing Lighttpd, Ruby on Rails, FastCGI, and MySQL on RedHat Enterprise Linux 5
Recently, I needed to configure a RedHat Enterprise Linux 5 box with lighttpd, FastCGI, Ruby on Rails, and MySQL. The box was subscribed to RHN, so I assumed a few simple commands like “sudo yum install lighttpd”, etc., would do the trick. Imagine my surprise to find that lighttpd, Ruby, gem, and FastCGI were all not available via RHN. I don’t know what the folks at RedHat are thinking by not including these packages, but it made me wish the box were running Debian Linux (which, for the record, has all of these packages available via aptitude).
Anyhow, switching to Debian was not an option, so I had to go back and do this the old fashioned way. The following is a quick walkthrough of the steps to install lighttpd, FastCGI, Ruby on Rails, and MySQL on RHEL 5. I am going to assume that you possess a reasonable level of Linux kung-fu if you are going to attempt this, so I won’t go too deeply into the individual details.
First, make sure your box is fully up to date, Linux-wise:
sudo yum update
Next, stop and deactivate Apache, which is running by default (I think) on RHEL 5:
sudo /etc/init.d/httpd stop
sudo chkconfig httpd off
Next, install necessary libraries and build lighttpd from the source code. I am adding some configuration options to ./configure to install files into the /usr/* directories, rather than the /usr/local/* directories as would be typical. This is a personal choice, so feel free to strip the options if you prefer to have your manually built applications in /usr/local…
sudo yum install pcre-devel zlib-devel bzip2-devel openssl-devel
wget 'http://www.lighttpd.net/download/lighttpd-1.4.20.tar.gz'
tar xvfz lighttpd-1.4.20.tar.gz
cd lighttpd-1.4.20
./configure --program-prefix= --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/usr/com --mandir=/usr/share/man --infodir=/usr/share/info --with-openssl
make
sudo make install
Next, create a non-root user and group that will run lighttpd. I typically use “www-data”, since I am a Debian guy, but feel free to use whatever you want. The important bit, of course, is that the user is something other than root for (hopefully) obvious reasons:
sudo /usr/sbin/adduser -s /sbin/nologin www-data
sudo /usr/sbin/addgroup www-data
Next, configure lighttpd and create necessary config and log files and directories. The contents of lighttpd.conf will depend on the location of your Rails application, among other things. I have included a simple Rails config below…but I assume if you are doing this you probably already have that…
sudo mkdir /etc/lighttpd
sudo echo > /etc/lighttpd/lighttpd.conf <<!
#
# Global config
#
server.port = 80
server.username = "www-data"
server.groupname = "www-data"
server.pid-file = "/var/run/lighttpd.pid"
server.errorlog = "/var/log/lighttpd/error.log"
server.indexfiles = ( "index.php", "index.html" )
accesslog.filename = "/var/log/lighttpd/access.log"
url.access-deny = ( "~", ".inc" )
server.modules = (
"mod_rewrite",
"mod_redirect",
"mod_alias",
"mod_access",
"mod_auth",
"mod_fastcgi",
"mod_accesslog"
)
var.your.app = "/www/path-to-your-app"
server.document-root = var.your.app + "/public"
server.error-handler-404 = "/dispatch.fcgi"
url.rewrite = ( "^/$" => "index.html", "^([^.]+)$" => "$1.html" )
fastcgi.server = ( ".fcgi" =>
( "localhost" =>
( "min-procs" => 4,
"max-procs" => 4,
"socket" => "/tmp/your.app.fcgi.socket",
"bin-path" => var.your.app + "/public/dispatch.fcgi",
"bin-environment" => ( "RAILS_ENV" => "development" )
)
)
)
!
sudo chown -R www-data.www-data /etc/lighttpd
sudo echo "LIGHTTPD_CONF_PATH=/etc/lighttpd/lighttpd.conf" > /etc/sysconfig/lighttpd
sudo echo > /etc/init.d/lighttpd <<!
#!/bin/sh
#
# lighttpd Startup script for the lighttpd server
#
# chkconfig: - 85 15
# description: Lighttpd web server
#
# processname: lighttpd
# config: /etc/lighttpd/lighttpd.conf
# config: /etc/sysconfig/lighttpd
# pidfile: /var/run/lighttpd.pid
# Source function library
. /etc/rc.d/init.d/functions
if [ -f /etc/sysconfig/lighttpd ]; then
. /etc/sysconfig/lighttpd
fi
if [ -z "$LIGHTTPD_CONF_PATH" ]; then
LIGHTTPD_CONF_PATH="/etc/lighttpd/lighttpd.conf"
fi
prog="lighttpd"
lighttpd="/usr/sbin/lighttpd"
RETVAL=0
start() {
echo -n $"Starting $prog: "
daemon $lighttpd -f $LIGHTTPD_CONF_PATH
RETVAL=$?
echo
[ $RETVAL -eq 0 ] && touch /var/lock/subsys/$prog
return $RETVAL
}
stop() {
echo -n $"Stopping $prog: "
killproc $lighttpd
RETVAL=$?
echo
[ $RETVAL -eq 0 ] && rm -f /var/lock/subsys/$prog
return $RETVAL
}
reload() {
echo -n $"Reloading $prog: "
killproc $lighttpd -HUP
RETVAL=$?
echo
return $RETVAL
}
case "$1" in
start)
start
;;
stop)
stop
;;
restart)
stop
start
;;
condrestart)
if [ -f /var/lock/subsys/$prog ]; then
stop
start
fi
;;
reload)
reload
;;
status)
status $lighttpd
RETVAL=$?
;;
*)
echo $"Usage: $0 {start|stop|restart|condrestart|reload|status}"
RETVAL=1
esac
exit $RETVAL
!
sudo chmod 755 /etc/init.d/lighttpd
sudo /sbin/chkconfig --add lighttpd
sudo /sbin/chkconfig lighttpd on
sudo mkdir -p /var/log/lighttpd
sudo chown www-data.www-data /var/log/lighttpd
Next, build and install Ruby from the source code. Again, I am adding configuration options to ./configure to install files into the /usr/* directories:
wget 'ftp://ftp.ruby-lang.org/pub/ruby/1.8/ruby-1.8.7-p72.tar.gz'
tar xvfz ruby-1.8.7-p72.tar.gzcd ruby-1.8.7-p72./configure --program-prefix= --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/usr/com --mandir=/usr/share/man --infodir=/usr/share/info --with-openssl
make
sudo make install
Next, install MySQL. This is the one step that RHN handles, thankfully…
sudo yum install mysql mysql-server mysql-devel
chkconfig mysqld on ; /etc/init.d/mysqld start
You should probably take this opportunity to change your MySQL root password, for (hopefully) obvious reasons:
/usr/bin/mysqladmin -u root password 'new password'
/usr/bin/mysqladmin -u root -p -h 'hostname' password 'new password'
Next, install FastCGI for use by lighttpd to spawn Rails dispatchers (again, using options for the /usr/* directories…):
wget http://www.fastcgi.com/dist/fcgi.tar.gz
tar xvfz fcgi-2.4.0.tar.gz
cd fcgi-2.4.0
./configure --prefix=/usr
make
sudo make install
Next, install RubyGems, so that we can grab Rails:
wget 'http://rubyforge.org/frs/download.php/45905/rubygems-1.3.1.tgz'
tar xvfz rubygems-1.3.1.tgz
cd rubygems-1.3.1
sudo ruby setup.rb
Next, grab Rails and install some other useful Gems:
sudo gem update --system
sudo gem update
sudo gem install rails --include-dependencies # installs Rails 2.1.2
sudo gem install mysql -- --with-mysql-config=/usr/bin/mysql_config # installs mysql-2.7
sudo gem install fcgi # installs fcgi-0.8.7
sudo gem install packet # installs packet 0.1.14
And, finally, if all has gone according to plan, you can bring up lighttpd and access your Rails application:
sudo /etc/init.d/lighttpd start
Hope this helps. Maybe some day soon this will be condensed to about five yum install commands via RHN…
Rails 2, Flex 3, and Form Authenticity Tokens
Recently, I was working with a Ruby on Rails application and I had the need to call a Rails controller method, with some parameters, from a remote Flex client. I would have thought that this would be a simple HTTP GET or POST to the Rails controller/method URL, using a Flex HTTPService object, with a tweak to the Rails method to render XML back to the client for parsing within Flex. However, Rails introduced the concept of form authenticity tokens in Rails 2.0, and these tokens are designed to block naive attempts to call Rails controller methods from outside of views rendered by Rails.
In simple terms, form authenticity tokens are one-time hashcodes that are generated as a hidden parameter for any form that is rendered by Rails. When the form is submitted, the hashcode is passed as a hidden parameter to the Rails controller, and Rails validates this hashcode to ensure that the form submission came from a view generated by Rails. This provides a measure of security against naive attempts to submit the form from other clients, since they will not have the proper hashcode needed to pass the Rails authenticity filter for the form submission. The specifics of the hashcode generation algorithm are covered elsewhere, but it suffices to say that they will resist uninspired hacking attempts, and it requires significant kung fu to bypass them without access to the Rails application.
In my case, I am not trying to hack the application — I just need to allow my Flex client to call my Rails methods. So I need to emulate the control flow of form generation in Rails, so that the view that kicks off my Flex client will contain a generated form authenticity token that can be passed to the Flex client as a startup parameter. There are 3 parts to this (or two if you want to condense parts 1 and 2):
- Store the generated form authenticity token for the Flex launch view in a javascript variable, so that it can be substituted intto the flashvars parameter of the Flex AC_FL_RunContent() javascript method. I chose to put this in the layout for the Flex launch view with:
<%= javascript_tag "const AUTH_TOKEN = #{form_authenticity_token.inspect};" if protect_against_forgery? %>
- Modify the call to AC_FL_RunContent() in the Flex launch view to include the form authenticity token. The line of code for this in the AC_FL_RunContent parameters list (if you are using my method of storing this is javascript as AUTH_TOKEN) is:
AC_FL_RunContent( [...] "flashvars","authenticityToken="+AUTH_TOKEN, [...] );
- I can now access the form authenticity token within Flex Actionscript code with a reference to:
Application.application.parameters.authenticityToken
Now that we have the form authenticity token in Flex, all that is left is to pass it as a parameter in the GET or POST to the Rails controller method. I found this last step to be surprisingly tricky. The Flex HTTPService object allows you to specify the parameters for a HTTP POST operation via an XML structure. Rails happily accepts and parses XML in POST operations, provided that the content type is set appropriately to application/xml. The tricky part is that the XML structure that is submitted by the Flex HTTPService object will be wrapped with a root XML tag of <request></request>, and all of the specified parameters will be contained within these tags. Rails will look for the form authenticity token as a root level tag named <authenticity_token>, and if it sees only a single root level tag of <request> (as sent by the HTTPService in Flex), then it fails the form authenticity test.
The workaround is to pass the form authenticity token as a URL parameter in the target URL of the HTTPService object, and to pass the other form variables within the standard request block of the HTTPService object, e.g.:
<mx:HTTPService id="httpService"
url="http://mysite/method/?authenticity_token={Application.application.parameters.authenticityToken}"
[...]>
<mx:request>
[...]
</mx:request>
</mx:HTTPService>
The result of this is that Rails sees two parameters in the form submission: an XML document with a root tag of <request>, and the form authenticity token with its proper name of <authenticity_token>. The form parameters are accessible via the XML document, and the form authenticity token is automatically found and validated by the form authenticity filter in Rails.
That’s it. Hope this helps.
ThreeD – A 3-D Rendering Engine in Java
|
ThreeD is an object-oriented rendering engine package written in Java. The ThreeD graphics pipeline is designed to support the transformation and visualization of Entities, each of which are subclasses of objects which encapsulate their own rendering and transformation algorithms and conform to a specific interface. The engine supports a switched, double-buffering scheme for the image buffer and the Z-buffer, using a support thread to clean buffers before use.
The current implementation contains a single Entity subclass: The Polygon class. The Polygon class renders polygons in three dimensions (using homogeneous coordinates) with perspective transformation using the Z-buffer for hidden surface removal. Each Polygon has a uniform RGB color, and the rendered color is adjusted by a primitive lighting component based on the dot product of the Polygon surface normal and the view vector. Polygons are rendered using an integer scanline algorithm. Anti-aliasing and texture mapping are not currently supported, but the class could be easily modified to support these (and other similar) features.
Performance notes for v1.0, circa 1996…
The performance is somewhat less than I had hoped for. The average performance of the Test applet shown above is (running via Netscape 3.0 on a PowerPC Macintosh, 200×200 applet geometry):
| Entities Rendered Per Frame: | 384 |
|---|---|
| Average Render Time Per Frame: | 0.75 seconds (+- 0.05 seconds) |
| Average Frame Rate: | 1.33 frames/second |
| Average Entity Rendering Rate: | 512 entities/second |
The Test applet writes real-time performance statistics to the Java console, so you can gauge the performance of your browser and machine. The multi-threaded environment causes a few wrinkles in screen update management, since there is no synchronous method to force an image frame screen redraw. The solution is to use synchronous locks on the buffers to control write access and preclude buffer modification until the buffer has been appropriately read. Consequently, a non-trivial fraction of the rendering period is consumed while waiting for the screen update to complete. The double-buffering strategy is meant to address this, but the lack of a fast array assignment method in Java reduces its effectiveness, as it takes a significant amount of time to reset an entire buffer to zero.
Performance notes for v2.0, updated in 2006 for Java 1.5…
Hardware and JVM performance has certainly come a long way in 10 years. I recently updated the code for Java 1.5, and I found that I had to throttle back the frame rate to get acceptable visual clarity – the engine can render a frame faster than the JVM can refresh the screen, even at the double size (400×400) of the current applet above. Java now provides more extensive double-buffering and synchronous screen update mechanisms, and these permitted me to remove many of the hacks I needed back in 1996 with the original implementation running via Netscape 3.0.
Here are some typical stats for the updated code on Firefox 1.5 using the Java 1.5 browser plugin (400×400 applet geometry):
| Entities Rendered Per Frame: | 384 |
|---|---|
| Average Render Time Per Frame: | 0.02 seconds |
| Average Frame Rate: | 40 frames/second |
| Average Entity Rendering Rate: | 38400 entities/second |
ThreeD is now hosted on Sourceforge at http://sourceforge.net/projects/java3d/. Both source and binary distributions of ThreeD v2.0 are available for download.
The Image MeshWarp Applet
|
This is an implementation of a meshwarping algorithm, using polygon scan conversion with texture
mapping to produce the warped image. On release of the mouse button after modifying the position of a vertex, the four affected polygons incident on the modified vertex are rendered with a relatively standard Bresenham incremental scanline conversion algorithm, using the initial fixed grid polygons of the image as textures to be mapped onto the new polygons. The algorithm is lossy, in that the texture mapping does not integrate areas of the source textures to produce pixel values. The texture mapping is also prone to distortion along the edges of adjacent polygons, which can produce unrealistic results. Vertex adjustments are reversible and idempotent.
Ultimately, it’s more toy than serious graphics tool…but fun nonetheless.
IncompatibleClassChangeError when deploying Axis 1.2-RC3 on WebSphere 5.1 with PARENT_FIRST classloading
I am deploying Axis 1.2RC3 in a war that resides within an ear. The axis jars are in the war/WEB-INF/lib dir. For various reasons related to the other contents of my ear, I must use PARENT_FIRST classloading. As many have reported, if you have PARENT_FIRST classloading on WebSphere 5.1 with Axis 1.2RC3, then there is a conflict between the version of saaj (1.1) that ships with WebSphere, and the version of saaj (1.2) that Axis 1.2RC3 expects, and you get the following exception:
java.lang.IncompatibleClassChangeError: class org.apache.axis.SOAPPart does not implement interface org.w3c.dom.Document
This issue can be resolved with PARENT_LAST classloading and a manifest which specifies the classpath…but for those of use who can’t use PARENT_LAST classloading, there is another option.
It appears that a single source code change to Axis will eliminate the issue altogether. The offending code is in org.apache.axis.message.MessageElement.addTextNode(), and can be patched as follows:
OLD CODE:
public SOAPElement addTextNode(String s) throws SOAPException {
try {
Text text = getOwnerDocument().createTextNode(s);
((org.apache.axis.message.Text)text).setParentElement(this);
return this;
} catch (ClassCastException e) {
throw new SOAPException(e);
}
}
PATCHED CODE:
public SOAPElement addTextNode(String s) throws SOAPException {
try {
Text text = new org.apache.axis.message.Text(s);
this.appendChild(text);
return this;
} catch (ClassCastException e) {
throw new SOAPException(e);
}
After making this change, Axis 1.2RC3 will run on WebSphere 5.1 with PARENT_FIRST classloading, and no exceptions will be thrown.
That’s it. Hope this helps.
Recent Posts
- Siri – a Shot Across Google’s Bow
- “Is colocation cheaper than using a cloud computing service to run the same workload?”
- “How Big is Amazon’s Cloud Computing Business?”
- “Android will run majority of smartphones by Spring”
- Amazon EC2 I/O Performance: Local Ephemeral Disks vs. RAID 0 Striped EBS Volumes
- “We create 5 exabytes every two days.”
- “Go Screw Yourself, Apple.”
- “You are not Google. (or: you don’t really need NoSQL…)”
- “The largest cloud providers are botnets.”
- Observed Performance of Amazon EC2 Instances
- Cloud Computing and Mobile Devices
- Time and Clock Issues in Windows-Based EC2 Instances
- My experimental local and real-time search engine is now available
- Entropy in Cloud Computing Applications
- How to Jailbreak iPhone 3.01