… this program finds the ulimit -u value, and then attempts to start that number + 1 SLEEP processes without waiting. You should find that before it can start that many SLEEP processes at the same time, some operating system errors will occur. On my Mac I get “sh: fork: Resource temporarily unavailable”. This is the type of error the dispatcher or proxy process will get when it attempts to start too many child processes.
Similarly on the open files side of things, the proxy processes and fglrun processes will open a number of files. Tools such as lsof allow you to see how many files are open by a process, wether they are the .42m files or the various log files that are being written.
Again a little test program can be written so that you can see what happens if your Genero application tries to open too many files at the same time …
# Program to exceed ulimit -n setting # DO NOT RUN on a production or live server, intended for use on private development server # Review the value for SAFETYNUMBER IMPORT os IMPORT util CONSTANT SAFETYNUMBER = 100000 MAIN DEFINE u base.Channel DEFINE ch DYNAMIC ARRAY OF base.Channel DEFINE filename STRING DEFINE i INTEGER DEFINE ulimitn INTEGER DEFINE step INTEGER DEFINE l_status INTEGER LET u = base.Channel.create() CALL u.openPipe("ulimit -n","r") LET ulimitn = u.readLine() CALL u.close() DISPLAY "ulimitn -n = ", ulimitn, " open files" LET step = ulimitn / 5 -- Just make a check that ulimitn is a sensible number -- If it exceeds the SAFETYNUMBER, then stop -- it maybe that you need to increats the safety number IF ulimitn >0 AND ulimitn <= SAFETYNUMBER THEN -- ok ELSE DISPLAY "Something has gone wrong, check safety number or result of ulimit -n" EXIT PROGRAM 1 END IF DISPLAY "ulimin -n = ", ulimitn, " open files" FOR i = 1 TO (ulimitn+1) IF i MOD step = 0 THEN DISPLAY i END IF TRY LET filename = os.Path.makeTempName() LET ch[i] = base.Channel.create() CALL ch[i].openFile(filename,"w") CATCH LET l_status = status DISPLAY "Status = ", l_status DISPLAY "sqlca = ",util.Json.stringify(sqlca) DISPLAY "Error occured with i = ", i EXIT PROGRAM 1 END TRY END FOR END MAIN
… this program finds the ulimit -n value, and then attempts to open that number + 1 files. You should find that an error will occur before it can open that many files. This is the type of error the fglrun process will get when it attempts to open too many files.
In both programs, note the comment in the code, don’t run this program on your live production server, this is intended for experimental use on your private development server where you are not going to inconvenience others.
In both programs note the SAFETYNUMBER, this is to protect in case the ulimit value is already set to a high or unlimited value.
The first question was “Why do my Genero applications running through the Genero Application Server stop after a while when there are a large number of users?”. This will often manifest itself in that on the first day you go live with Genero Application Server and you have not had that number of users on your test server. For the first users the system will be fine but then after more and more users are on, something will stop as the ulimit -u limit is reached. The dispatcher will be restarted and same thing will happen, for the first users it will be good but eventually an error will occur. It maybe that this is around the 300-500 user mark when the ulimit -u value of 1024 is being reached. You’ll ask yourself why did this not occur on the identical test machine and chances are you never tested with that many users at once. You may ask why this did not occur when using direct connection instead of via Genero Application Server and it will be because there isn’t the fastcgidispatch process launching everything and there isn’t the proxy process increasing the proxy count.
I have deliberately avoided advising you how to change these limits, it can be via the ulimit command itself for the current process, or via editing values in a file, either for all users or for the user used by the dispatcher process. This would normally be something you’d change in conjunction with your system administrator.
As I often say, Genero (as in Genero Application Server or Genero Studio) can’t break the rules of operating systems, isn’t doing magic, it is executing commands on the command line and is subject to the rules of the operating systems. ulimits refers to the boundary lines imposed by the operating system and you may need to move them to cater for your Genero application.