Recent twitter entries...

  •  

Nginx error with Tomcat – upstream sent too big header while reading response header from upstream

Posted by Nitai | Posted in Development, linux, open source | Posted on 02-02-2010

0

Nginx is our favorite web server currently as it is fast, lean and easy to configure. Performance is just outstanding and if you haven’t take a look at it.

One thing that I noticed while we deployed Nginx with Tomcat is that their default size for the buffers are very low. Don’t know why a 4K buffer size would be sufficient. If you keep it at their default size and might have a heavy duty Tomcat app then it could be that you will see error messages like:

upstream sent too big header while reading response header from upstream

The remedy for this error is to set the parameters for higher values for the proxy_buffer* parameters as we have done and now all is back to normal. Here are the current settings that worked well for us;

proxy_buffer_size   128k;
proxy_buffers   4 256k;
proxy_busy_buffers_size   256k;

Setting up Apache2, PHP and MySQL on MacOS X – the easy way

Posted by Nitai | Posted in Apple, Development, open source | Posted on 23-01-2010

0

Today I set out to get MySQL and PHP setup on my MacBook Pro. Since Apache2 already comes with MacOS X (mine is Snow Leopard and yours should be too!). Now, the funny thing is that I first searched on the web how to best install PHP and MySQL.

Surely, I came across MAMP (a package that gets you Apache2, PHP, MySQL and a couple libraries) in a nice one click application and some others. Being the guy who rather has things separated and controllable, I quickly shined away from those. Thought, I gave MAMP a try, but could not get MySQL to listen to anything else then the internal Apache2 server from the MAMP package (but guess that is another story and I’m really not doing this the first time.). Anyhow…

I then looked into getting Apache2, MySQL and PHP with MacPorts. Thought MacPorts has proven to be perfect in such circumstances, I had a hard time (and it took very long) to get this setup up and running. I’m sure, some of you have had successful installs and all works great, but at the end it did not work for me. There are even more instructions to get PHP running, with a lot of tweaking and such, but to be honest in the end…

Really the simplest and most straightforward method to get Apache2, MySQL and PHP running on MacOS X is;

MySQL

Now, this is really no brainer. All you need to do is to go to http://www.mysql.com and download the recent release. Within the download image you will find a nice installer and Preference pane which lets you start/stop MySQL. If you want to go all GUI, then also download the GUI tools from MySQL.

Apache2/PHP

The probably easiest of it all. Since MacOS X already comes with Apache2 and PHP all you need to do is to enable it. Thought, PHP is disabled in the httpd.conf, all there is to do is to edit httpd.conf and uncomment the mod for the php library.

That’s it!

Nothing to install, (almost) nothing to configure. Simple and easy.

Subversion: Switch to a new location

Posted by Nitai | Posted in Development, open source | Posted on 11-01-2010

0

We just moved some of our subversion repositories to a new server and with it changed the URL’s. Now, in order to change any existing checkout version to grab updates from the new subversion server we issued the simple “switch” command from subversion as in:

svn sw svn://svn.domain.com/trunk .

The problem with this was that the system complained with:

svn: 'svn://svn.domain.com/trunk'
is not the same repository as
'svn://svn.newdomain.com/trunk'

The correct syntax to relocate any existing checkout copy is to use the “–relocate” switch as in:

svn sw --relocate svn://svn.domain.com/trunk svn://svn.newdomain.com/trunk

And we are back in business…

Wordpress, PHP error and updating PHP on CentOS 5.x

Posted by Nitai | Posted in Development, open source | Posted on 21-12-2009

0

Mind you, I’m no PHP guru and I only have PHP installed on our server because of our Wordpress blogs. that said, I was facing an ugly PHP error today and somehow we could not load the Wordpress admin pages anymore of it. The error was:

PHP Warning:  Module 'fileinfo' already loaded in Unknown on line 0

After searching for a solution I thought it would be best to update PHP along the way. Thus I found this nice description on Webtatic Blog which describes how to update to the latest PHP version on CentOS 5.x (since the default repro only brings you 5.1.6).

Low and behold, after updating to the latest PHP version, it will give you 5.3.1, the error was gone and Wordpress Admin pages are back to normal.

Solutions for Oracle ORA-28002 and ORA-27101

Posted by Nitai | Posted in Development, linux, open source | Posted on 11-12-2009

0

Running an Oracle database is great, because it is stable and just runs, but sometimes you are hit with unexpected errors when you restart your machine. Thought, I know Oracle quite well, I’m always surprised at some things. Here are two errors (and solution) I was just confronted with:

ORA-27101: shared memory realm does not exist

To be honest, I don’t know why I was confronted with this error, since we haven’t changed anything to the machine or to the environment variables. Metalink suggest to check that the ORACLE_HOME and ORACLE_SID are correct. Funny thing is that this system runs for over a year without a change to the path, nevertheless I checked the ORACLE_HOME path and sure enough I had trailing slash at the end.

So, the solution was to see that ORACLE_HOME does NOT have a trailing slash.

echo $ORACLE_HOME
/opt/oracle/product/11/ <--- WRONG!

echo $ORACLE_HOME
/opt/oracle/product/11 <--- CORRECT!

Remember to log out of your current shell session in order to reapply the new settings.

ORA-28002: the password has expired

Now, this error caught be even worse, because it happened right within a production environment. Also, here I was unaware of this setting. In any case, Oracle seams to want you to reset your password after one year or so. In case, you want to disable this on a user without changing the users password you need to issue the following commands in sqlplus:

alter profile {user} limit password_verify_function null;

This will set the verification for this user to null. If you want to do this for every user in your system you would use this (this is applied to the DEFAULT profile):

alter profile DEFAULT limit password_verify_function null;

Once done you can then reset the password for the user with the same password or with another one with:

alter user {user} identified by {password};

Exit sqlplus and your changes should have been applied.

ImageMagick and Ghostscript playing nice with web applications

Posted by Nitai | Posted in Development, linux, open source | Posted on 20-11-2009

1

I had to spent way too much time the last time to set up ImageMagick and Ghostscript together that I simply jump right in so you don’t have to waste time on this, like I did.

Creating thumbnails and images from a PDF is one of the most used features within Razuna. But the other day, when we run the normal convert command we were faced with an error message within our web application of:

Error: Uncaught exception ‘ImagickException’ with message ‘Postscript delegate failed....: No such file or directory

Of course, the first thing I did was to see if we have a error in the shell. Strangely, there was NO error when running the same command in the shell! Needless to say, I spent many hours looking for a solution, trying out different things from reinstalling ImageMagick and Ghostscript to looking all over the web for an answer.

In the end, it boiled down to that web applications (be it CFML, PHP, etc.), are looking for a different executable path then when you run command from the shell. Since Ghostscript installed the executable in the “/usr/local/bin/gs” path the web application could not find it. So the solution to “hours of pain” was to simply create a symbolic link to the “gs” library in the “/usr/bin” directory with:

ln -s /usr/local/bin/gs /usr/bin/gs

Hope this helps the next seeker!

SQL Paging with Oracle, MySQL and MS SQL

Posted by Nitai | Posted in Development | Posted on 15-11-2009

0

I have to admit, I love databases. To hold data in a central place and be able to hook up any application from wherever you are to it, is just convenient to say the least. But there are moments I wish that vendors would “follow” a standard. I guess, when it came down to paging, every vendor thought his way of doing it is the best. No wonder, paging is the one SQL syntax that differs the most. Thus I dedicate, this post to paging and outlining the difference, meaning the solution to achieve it with Oracle, MySQL and MS SQL.

Paging with Oracle

Oracle knows of the internal column “ROWNUM”. Thus with “rownum” we can page trough the records, but have to do this with subselects. Look at the example;

SELECT rn, columns
   FROM (
	SELECT ROWNUM AS rn, columns
		FROM (
			SELECT columns
			FROM table
			WHERE id = 1
			AND something = 2
			ORDER BY columns
			)
		WHERE ROWNUM <= 10
	)
WHERE rn > 0

In this example we select the first 10 records, if you want to select the next 10 records then you will need to increase the values for “ROWNUM” with 20 and for “rn” to 10.

Paging with MySQL

There is one thing I like MySQL, that is the “LIMIT” clause. Not only can you limit how many records to retrieve, but can also add a offset to the limit clause. Thus we can do very elegant paging with “LIMIT offset, number”. Again, we limit the query below to return us 10 records:

SELECT columns
FROM table
WHERE id = 1
AND something = 2
ORDER BY columns
LIMIT 0, 10

Now, this will return the first 10 records. So, what if we want to do a paging here? Actually, quite easy, all we have to do is to increase the offset by one. It is important to know, that the offset start with a “0″ (think of it as the FIRST page), thus if we want to retrieve the next 10 records we use the limit with “LIMIT 1, 10″. This will give us the next 10 records. Elegant, no?

Paging with MS SQL

Now paging with MS SQL took me some time to figure out, since some said that MS SQL know of “rownumber” as well, but honestly, I couldn’t get anything to work with it. So the most easy method to do paging in MS SQL is to use the TOP syntax combined with a NOT IN subselect. Makes sense, right? Here is the example;

SELECT TOP 10 columns
FROM table
WHERE xxxx
AND id NOT IN (
	SELECT TOP 0 id
	FROM table
	WHERE xxxx
	)

As in the Oracle example above, if you would like to retrieve the next 10 records you would have to increase the TOP values by 10 records, meaning the FIRST TOP would have a value of 20 and the inner select TOP a value of 10.

So here you have it. How paging works for Oracle, MySQL and MS SQL.

FFmpeg — here we go again

Posted by Nitai | Posted in CFML, Development, linux, open source | Posted on 12-11-2009

0

My two other posts on FFmpeg entitled “Installing ffmpeg on CentOS 5” and “SELinux with ffmpeg” already explained in deep how to get FFmpeg up and running.

Unfortunately, today one of our servers just reported a plain:

ffmpeg: error while loading shared libraries: libfaad.so.0: cannot open shared object file: No such file or directory

when trying to run any ffmpeg command. Now, what was that and foremost why? Especially, since it worked 2 days ago. I guess, “someone” must have done some update. In any case, it was a good lesson to reinstall ffmpeg and bring all libraries up to date.

So, since my last installation, some things must have obviously been changed behind the scenes, because when I tried to configure ffmpeg with the same commands I run into another error which was:

libx264 version must be >= 0.78

Even thought I installed x264 from the latest GIT repository it still showed me the above error! Looking around the Videolan.org website I saw that the nightly snapshots differ in size a lot. So, either their nightly script is broken or something else is going on. In any case, I went with the x264-snapshot-20091031-2245 one.

Configured and installed it. Then did a “ldconfig -v” (in order to see that it really took the latest one) and went on with the ffmpeg configuration (I took the latest code from SVN (Revision 20525)) and low and behold, everything compiled and installed without problem.

Important: Update your libraries after the installation of ffmpeg again with “ldconfig”! Else you will get the “ffmpeg: error while loading shared libraries: ….” error again.

Tomcat and directories

Posted by Nitai | Posted in CFML, Development, open source | Posted on 11-11-2009

0

I run into a nasty issue the other day with “alike” directories in my web application.

That is, I had a copy of a “WEB-INF” directory called “WEB-INF–”. While my web application did not bother with copy of the WEB-INF directory, Tomcat threw all kinds of errors in the catalina.out log file. Also, I had some issues with Tomcat restarting and strange Java error messages, like:

Java.lang.NullPointerException
com.nary.util.Localization.convertCharSetToCharEncoding(Unknown Source)
com.naryx.tagfusion.cfm.engine.cfEngine.getDefaultEncoding(Unknown Source)
com.naryx.tagfusion.cfm.engine.cfFormData.(Unknown Source)
com.naryx.tagfusion.cfm.engine.cfFormData.(Unknown Source)
com.naryx.tagfusion.cfm.engine.variableStore.(Unknown Source)
com.naryx.tagfusion.cfm.engine.cfSession.(Unknown Source)
com.naryx.tagfusion.cfm.engine.cfEngine.service(Unknown Source)
com.naryx.tagfusion.cfm.cfServlet.service(Unknown Source)
javax.servlet.http.HttpServlet.service(HttpServlet.java:717)

After browsing the log files, I saw that Tomcat tried to read the directory and threw a bunch of critical errors while reading those copied directories. I guess, Tomcat got totally confused and tried to overload libraries.

Lesson learned; Never have any copy of the WEB-INF folder within the web application again.

Upgrade to most recent ImageMagick version on CentOS 5.x

Posted by Nitai | Posted in Development, linux, open source | Posted on 10-11-2009

1

My favorite choice for running a Linux Server is CentOS, since it is based on the RedHat distribution you can rest assured you will get a top notch enterprise offering and stability. As with all things “enterprise” the priority is on stability and security and not on the latest code releases. This works 99% of the time, but sometimes you still need some update.

In the case of ImageMagick, CentOS comes with version 6.2.8, it was a bug that was fixed with PSD conversion and thus I needed to get the latest version installed. So, here are the steps to install ImageMagick 6.5.7 on CentOS 5.x. Mind you, that you will loose the internal patch upgrading from yum, but all you need to install to the next version is just to follow these steps again.

Uninstall current version
Uninstall the current version with:

yum erase ImageMagick*

This will uninstall ImageMagick 6.2.8 and if you have any other versions installed, like the devel one.

Install the needed dependencies
ImageMagick depends on a couple of additional libraries to convert to different formats. Let us just make sure, that they are all installed with:

yum install tcl-devel libpng-devel libjpeg-devel ghostscript-devel bzip2-devel freetype-devel libtiff-devel

Download and extract latest ImageMagick version
You can always get the latest ImageMagick version directly from their website. Code below will download and extract the file.

wget ftp://ftp.imagemagick.org/pub/ImageMagick/ImageMagick.tar.gz
tar xcvf ImageMagick.tar.gz
cd ImageMagick-6.5.7-5

Configure and make ImageMagick
With the below configure command we are configuring ImageMagick with the most needed options. Feel free to adjust it to your needs. As always issue a “–help” to see all the available options.

configure --prefix=/usr/local --with-bzlib=yes --with-fontconfig=yes --with-freetype=yes --with-gslib=yes --with-gvc=yes --with-jpeg=yes --with-jp2=yes --with-png=yes --with-tiff=yes

Wait until configure has finished. At the end you will see all the enable options. When you think all went well issue:

make

Now is a good time to make yourself some coffee or continue coding your next big killer application because make will take some time to finish. When it’s done, issue:

make install

That’s it! You are done. Wasn’t so bad, was it? Check with:

convert --version

That ImageMagick is properly installed and that you got the current version up and running. If all went well you should see something similar to this:

Version: ImageMagick 6.5.7-5 2009-11-08 Q16 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2009 ImageMagick Studio LLC