Ask Reuben

Upgrading

How do I upgrade? 

Any tips for upgrading?

Last week saw the release of Genero Enterprise 4.00.  Whenever there is a release there are a number of support questions asking how to upgrade, I thought I’d share some tips

Make Upgrading a Habit

In my opinion the number one reason customers struggle with upgrades is because they don’t do them often enough.  We do a major release approximately every 18-24 months, and numerous maintenance releases in between.

If you skip a major release or two, it can be 4-6 years plus between upgrades for a major release.  In that time personnel can move and the corporate knowledge of how and what to do during  upgrade has been forgotten.

Skipping releases also means that when you do multiple upgrades in one jump, it is a bigger more painful leap as you have multiple upgrade notes to navigate.  It becomes a self-perpetuating negative experience.

There are other reasons you should not skip upgrades.  Being on the latest release gives you …

  • the latest functionality
  • latest versions of third-party libraries including those used for security
  • latest versions of operating system and database support
  • minimal disruption if you need to take the latest release in a hurry i.e. security update.

Finally something that is very frustrating is to spend many hours analysing an issue only to discover that not only has the issue being previously reported a year or two ago, but that it has been resolved in a subsequent maintenance release.   You are paying for the maintenance but you are not taking advantage of what you have paid for.

Identify the Type of Upgrade.

The Genero version numbering scheme is X.Y.Z. e.g. 4.00.03  (this numbering scheme may also be referenced as M.FF.BB)

A maintenance release is where only the Z or BB number changes e.g. 3.20.15->3.20.16.  A maintenance release does not require a recompilation as per this topic on p-code compatibility.

The key point to note is that at a production site, you can install a later Genero BDL maintenance release and point FGLDIR at this later release.  With a maintenance release you do not need to recompile  and redeploy your recompiled Genero application.

Similarly a maintenance release of one Genero product does not typically need to be accompanied by the maintenance release of another Genero product.  You can typically do a GDC, GBC or GAS maintenance release without having to do a corresponding BDL maintenance release.  With a major release where X.Y (or M.FF) changes, then you are typically updating all our products to keep them in sync.

Do Your Reading

There are many places you can find out what is in a release.

There is the announcement notice for a high level view

Each product documentation has two sections to read.  There is the New Features guide which is the interesting section with the new syntax and functionality.  The Upgrade Guide is the must read that contains details where you may have to change your code.  Read the equivalent sections in every products documentation.  I have given the links to the BDL documentation, but in the documentation page of the website you will also find documentation for other products, so don’t forget to read the GAS, GST documentation etc.

The Issue Tracker also contains notes about what has been fixed in each release.  A prudent developer will read these and note where they should add extra QA effort.  Note that you can use the search here to identify changes between individual releases, this is useful during maintenance releases.  For example, the search page constructs a URL such as https://4js.com/support/issue/?product=FGL&stat=CLOSED&fixed_in=3.20.12-3.20.13 which lists closed issues in FGL between 3.20. 13 and 3.20.14.

For a major release the two documentation sections are the primary source of information.  For a maintenance release, the Issue Tracker is the primary source of information about what is included in the maintenance release.

If skipping releases, make sure you also read the information for the intermediate releases.

Participate in Early Access Programs

Participating in Early Access Programs (EAP) gives you access to the releases ahead of general availability.  Participation is encouraged.  A previous Ask-Reuben gave reasons for participating in the EAP.

Default Installation Directories

When installing our products, do not use our default directory names but instead consider adding the version number in the directory names.  For example, instead of /opt/fourjs/fgl, install the 4.00.03 release into opt/fourjs/fgl/4.00.03

If I see …

opt/fourjs/fgl/3.10.15
opt/fourjs/fgl/3.20.13
opt/fourjs/fgl/4.00.00

… I normally find that that organisation is more organised and better placed than an organisation whose directories are named  …

/opt/fourjs/fgl
/opt/fourjs/fgl.new
/opt/fourjs/fgl.new2
/opt/fourjs/fgl/old

You also need to keep the versions separate so that you can run and compare different versions.  If you have a single version in /opt/fourjs/fgl and do you an upgrade by overwriting that installation, consider how you would then respond if someone says “the new version is behaving differently than the old version”, how do you run the old version to compare?

Installation Order

The installation order is influenced by what products ask questions regarding other products during installation.

For that reason, I install FGL, then GRE, then GAS as GAS asks for a default FGL and GRE to use.

On OSX I also install GDC before GST.  This is because OSX registers the location of each application and I want the location of the GDC to be the standalone copy of the GDC and not the GDC bundled inside Genero Studio.

Root Directories

Our products are designed so that they are all beneath a root directory and environments can be defined relative to a root directory.  $FGLDIR you will hopefully recognise as the root directories of the BDL product and that $FGLDIR/envcomp is a script that sets the minimum environment necessary to run the product.  Genero Application Server has $GASDIR and $FGLASDIR/envas, Genero Report Writer has $GREDIR and $GREDIR/envgre respectively, and similar for other products.

Note: for simplicity I will reference environment variables $FGLDIR (*nix, OSX), %FGLDIR% (Windows) as $FGLDIR, Windows users can remove the $ and add the %.

To switch Genero versions, your environment should be configured so that you simply change $FGLDIR, $FGLASDIR, $GREDIR etc. and point at a different version.  You can also use a technique that involves symbolic linking.  Both techniques are discussed here.

To do the same with your application, you should have a similar structure.  Have a root environment variables e.g. $MYDIR that points at a directory beneath which all your Genero application files are placed.

You can configure your environment so that $FGLDIR etc points to the Genero product version you want to run, and that $MYDIR points to the version of your application that you want to run.

To separate static program files and dynamic data you would probably have another environment variable that points at a different directory beneath which are your data and configuration files are placed.  In Windows these are files you see in the %APPDATA% folder, whilst in Linux/OSX these are files you might see beneath $HOME e.g. $HOME/.fourjs or in other directories such as Library/Application Support/FourJs in OSX.

Ports and Data Directories

As well as having different values for $FGLDIR to easily switch between different version, you will need to consider different values for data directories and ports.

For Genero Application Server, note the different ports and data directories, you need to define to keep the separate instances apart.  For httpdispatch, you might assign ports 6394,6395, 6396 to three different versions of the GAS and set TCP_BASE_OFFSET accordingly, either in your main configuration or via use of the -E argument.  Similarly for each version  TCP_ADMIN_PORT and the value of res.appdata.path.

You can run multiple instances of the GDC on the same PC.  Note the command line option to start multiple instances of the GDC with different ports.

You will need to assign different ports to the different versions and configure accordingly.  For example, you might assign ports 6400, 6401, 6402 to three different versions of the GDC, and have your environments set $FGLSERVER accordingly.  For .gdc files, you can use @FGL in the short-cut configuration to do this for you.

Other Configuration

In order to facilitate quickly moving between GBC installations I do the following.

With each GBC installation being installed in a unique directory beneath /opt/fourjs/gbc I make the following changes to as.xcf.

  1. define a new resource res.gbcdir as /opt/fourjs/gbc (the directory where I install all my gbc configurations)
  2. amend GBC_LOOKUP_PATH so that $(res.gbcdir) is the second value in the path
  3. change res.gwc-js to the latest version e.g. 4.00.03

Doing this means that when running I am using the last GBC by default and I can quickly switch to another GBC by adding ?gbc= to the URL and this will use the specified gbc.

If using the GDC and Universal Rendering, then $FGLGBCDIR can be used to point to the desired gbc in /opt/fourjs/gbc.

Other changes I make to my standard as.xcf to facilitate development are to

  1. set res.access.control to ALL for ACCESS_CONTROL in my development environment.
  2. set res.uaproxy.param to –development so that the proxy runs in development mode.
  3. add ALL to res.log.categories_filter to set CATEGORIES_FILTER so I get more information in the logs.

Unbundle

Genero Studio comes shipped with the latest versions of each product that are bundled in the Genero Studio directories.  It does not come with a mechanism where you can update these bundles installations with the latest maintenance release.

My tip is to not rely on these bundled installations, download the individual installations and configure your Genero Studio configurations to use them.  I keep the default Studio configuration and use these as a reference when creating my configurations.

This also helps when it comes to deployment as you will not deploy Genero Studio to end-user sites.  The skill of downloading, installing, and configuring to use individual installations is a skill you will need.

This technique of unbundling can also be applied to other Genero products that are bundled inside Genero downloads.  For example I unbundle Genero Ghost Client which is distributed inside $FGLDIR/testing_utilities and place it in /opt/fourjs/ggc/ and set $GGCDIR to reference that directory.  Similarly I download GBC separately and place it in /opt/fourjs/gbc/ in conjunction with some GAS configuration so that I can reference different GBC with ?gbc= in the URL or $FGLGBCDIR=/opt/fourjs/gbc/

Overall I can quickly switch between the various versions with a change in environment variable or similar.

Licensing (Use Four Js License Manager)

I recommend the use of Four Js License Manager in a development environment.  Each Genero installation can share the same license.  At install time, it is simply a case adding to each installations fgllicense file (Genero 4.00 and FLM 6.00). (or fglprofile for versions prior) the license number and the server and port details of the running Four Js License Manager daemon.

Update Sources

Source files that save internally as xml typically need to be upgraded to the latest version.  For example, gsform -c will update 4fd form files, gsreport -c will update 4rp report files.  Internally these files are xml and what this will do is make any changes caused by changes in the xml structure such as changing of node name, attribute name, attribute values.

As a general rule of thumb, where the file is edited graphically, there should be a command to update the file.  For files that are edited in text editor (.4gl, .per, .4ad, .4tb, .4tm, .4st, .4sm), there will not be.

One trap to avoid is where you have made changes to files we supply, such as .js / .scss GBC customisation files, $FGLASDIR/etc/as.xcf.  A common mistake is to simply reuse these files.  What you need to do is reapply our changes.  It is easy to explain with a file like $FGLASDIR/etc/as.xcf.  Between 3.20 and 4.00 we have added some additional entries, changes some values.  If you copy your modified 3.20 as.xcf into your 4.00 area you will miss those additional entries and changes values.  Use diff and patch to identify changes and reapply to the latest standard.

QA, Test

To validate an upgrade, you need to be in a position to run some tests that demonstrate that the functionality of the application on the different versions is the same.

These can either be automated using tools such as Genero Ghost Client, and/or be a suite of well documented manual tests.

The important thing is being in a position to assure yourself that your application still runs as expected and that you can be confident deploying it to your end users.

Deploy

Once you are ready to deploy to another server, wether QA or production, then very simplistically …

Install the required Genero products on the new server installing in a unique directory

Install your applications files on to the new server in a unique directory

Edit configurations so that …

  • $FGLDIR etc point to new version of our product
  • your $MYDIR points to the new versions of your application
  • other ports, directories point to new unique values

If necessary deploy files to the end user PCs if required, this typically means the GDC.

Deploying GDC

One of the biggest hurdles in any upgrade is deploying the Genero Desktop Client (GDC), particularly in a commercial environment.  It is something that I have found medium sized end-users struggled with.  Small sites would install GDC manually on each PC whilst larger sites typically had other applications that also needed installing on PC’s and so had corporate policies and practices on how to to do this.  Medium sites fell somewhere in-between.

My stock answer used to be, how did any other applications get installed and upgraded on the end users PC? gdc.exe is an application just like word.exe, outlook.exe, use the same technique.

There are some things you should note:

  • Just as browsers silently update themselves, we added in Genero 3.10 the ability for the GDC to also have its update managed automatically.  If you are not aware of this, read about Auto-update here.   Essentially a Genero application, typically a main menu or perhaps a dedicated update program that you instruct the end user to run, checks the version of the GDC it is connected to, and if it is not the latest, executes a front-call that downloads an update for the GDC and installs it.
  • With the 3.10 and 3.20 releases of GDC, effort was made so that these releases could be used with earlier 3.X release as detailed here.
  • You can run your Genero application using the Genero Browser Client (GBC) instead of the Genero Desktop Client, and not have to worry about how to deploy GDC.  Use the fact that  the browser is already there and you don’t have to explicitly deploy or manage it.   For an end-user it means giving them a different URL to run that points at the later version of your application.   Just be aware that using GBC and not GDC is not always the panacea you may think it is.  Browsers have rules they must adhere to, you need to consider the gains you make on simplicity of deployment against the loss of things like desktop integration, typeahead.

Conclusion

I have nearly finished the article and haven’t tackled the how.  The important thing is about having your development environment configured in a way that you have multiple development environments.

Wether you are an ISV, Service Provider, or an Enterprise customer, I’d expect you to have the ability to run your application on at least three different Genero versions at any given time.

  • The current version(s) your end users are using
  • The previous version in case the customer raises an issue and you need to compare against what it did previously
  • The next version you are working on for a release on a date in the future

If you have that in place already, then adding another version, wether a major or maintenance release becomes a simpler task.

Upgrading

With all the above in place, how to upgrade to a major release then becomes the following …

  1. Participate in the Early Access Program.
  2. After the official date of a new release, read the New Features and Upgrade Guide sections of the documentation for the Genero products you use.
  3. Download GST and also the various individual packages FGL, GDC, GAS, GBC, GRE, GMI, GMA etc.
  4. Install these products into their unique directories that includes the version name e.g. /opt/fourjs/fgl/4.00.03
  5. I do an additional step here, I extract any product that is shipped inside another product.  At the moment I do this for Genero Ghost Client.
  6. License the BDL installation by editing its fgllicense (fglprofile before 4.00).
  7. Edit $FGLDIR/etc/as.xcf or create your GAS configuration file with changes to facilitate multiple configurations and your development.
  8. If using Genero Studio, edit your configurations so that you have entries pointing to these new installations.
  9. If using command line, edit your development environment configuration so that you have environments that point to $FGLDIR, $FGLASDIR, etc of these new installations.
  10. Repeat these configuration steps for a new version of your application and your applications own root directory $MYDIR (replace MY with your application name).
  11. Using configuration that point to the new version of your application, and the new version of Genero, check out your source files, perform the automatic update of your source files and do a complete recompile of your application.
  12. In conjunction with the Upgrade Guide, review code that no longer compiles.
  13. In conjunction with the Upgrade Guide, make any recommended changes to your code base.
  14. Run a standard suite of QA tests on your application and verify that your application performs as expected.
  15. At this point you are now ready to start coding on any new functionality in your application, incorporating any New Features we have to our product into your product.
  16. Code, test, deploy