Ask Reuben

Mapping curl to httpRequest

How can I interact with a Web Service? 

A Web Service API documentation has code examples but not in Genero.

When coding a request to a third party Web Service, it would be nice if they follow standards and provide an OpenAPI specification that you can use fglrestful to generate 4gl code (RESTful), or if they provide a WSDL specification that you can use fglwsdl to generate 4gl code (SOAP).

Unfortunately not all Web Service providers do this.  What you may find is that their API documentation provides helper classes or code examples in various programming languages but not Genero.  You have to then interpret their documentation to write your 4gl code.  As an example, the Twilio SMS API has some libraries and articles for various languages but not Genero.   They might have some code snippets using curl, as the Twilio article does…

Missing a language you like? If you follow the conventions in our curl code snippets throughout the documentation, you can translate the API call to your programming language of choice.

curl -X POST "https://api.twilio.com/2010-04-01/Accounts/$TWILIO_ACCOUNT_SID/Messages.json" \
--data-urlencode "Body=This is the ship that made the Kessel Run in fourteen parsecs?" \
--data-urlencode "From=+15017122661" \
--data-urlencode "To=+15558675310" \
-u $TWILIO_ACCOUNT_SID:$TWILIO_AUTH_TOKE

Your programming exercise then becomes how to map the various curl options to 4gl methods.

What I recently put together was an example in our Github repository ex_ws_input_methods that took a simple function and showed many different ways that it could be exposed.  For each of these I generated client code but also illustrated what the equivalent curl command was to call that web service from the command line.

At the time of writing, the program illustrates 7 different ways that a function can be exposed as web service, 4 using GET and 3 using POST.  To see the relationship it is a case of finding how the function is exposed on the server side, find the generated client code, and find the curl example in the README.

If you compile and run the server program (in the server folder), you can use the various curl examples in the README to call the Web Service from the command line.  The code in the client program (in the client folder) was generated via fglrestful, and you can also run that program to call the web services in the many different formats.  If you set FGLWSDEBUG when you call the web service (either via 4gl code or curl), you can see what is being passed backwards and forwards.

So first in server/ex_ws_input_methods.4gl I find a function that exposes a simple function.  The code below is from the example using the WSQuery attribute setting that signifies to pass the function arguments via the query string.

FUNCTION add_wsquery(x INTEGER ATTRIBUTES(WSQuery), y INTEGER ATTRIBUTES(WSQuery)) ATTRIBUTES(WSGet, WSPath = "/Add_wsquery")
    RETURNS INTEGER

    DEFINE z INTEGER

    LET z = x + y
    RETURN z
END FUNCTION

In client/ex_ws_input_methods.4gl I have the code that was generated via fglrestful.  If the Web Service you want to connect to does not provide an OpenAPI specification, this is the type of code you have to write by hand.  Once you have this code managing the http request and response, you can IMPORT FGL  this module and call the web service exposed as a function e.g. …

IMPORT FGL ex_ws_input_methods_client AS calc
...
CALL calc.add_wsquery(?, ?) RETURNING wsstatus, result

The generated code, or the code you might have to write by hand is …

# Operation /Add_wsquery
#
# VERB: GET
# ID:          add_wsquery
#
PUBLIC FUNCTION add_wsquery(p_x INTEGER, p_y INTEGER) RETURNS(INTEGER, INTEGER)
    DEFINE fullpath base.StringBuffer
    DEFINE query base.StringBuffer
    DEFINE contentType STRING
    DEFINE headerName STRING
    DEFINE ind INTEGER
    DEFINE req com.HttpRequest
    DEFINE resp com.HttpResponse
    DEFINE resp_body INTEGER
    DEFINE txt STRING

    TRY

        # Prepare request path
        LET fullpath = base.StringBuffer.create()
        LET query = base.StringBuffer.create()
        CALL fullpath.append("/Add_wsquery")
        IF p_x IS NOT NULL THEN
            IF query.getLength() > 0 THEN
                CALL query.append(SFMT("&x=%1", util.Strings.urlEncode(p_x)))
            ELSE
                CALL query.append(SFMT("x=%1", util.Strings.urlEncode(p_x)))
            END IF
        END IF
        IF p_y IS NOT NULL THEN
            IF query.getLength() > 0 THEN
                CALL query.append(SFMT("&y=%1", util.Strings.urlEncode(p_y)))
            ELSE
                CALL query.append(SFMT("y=%1", util.Strings.urlEncode(p_y)))
            END IF
        END IF
        IF query.getLength() > 0 THEN
            CALL fullpath.append("?")
            CALL fullpath.append(query.toString())
        END IF

        # Create request and configure it
        LET req =
            com.HttpRequest.Create(
                SFMT("%1%2", Endpoint.Address.Uri, fullpath.toString()))
        IF Endpoint.Binding.Version IS NOT NULL THEN
            CALL req.setVersion(Endpoint.Binding.Version)
        END IF
        IF Endpoint.Binding.Cookie IS NOT NULL THEN
            CALL req.setHeader("Cookie", Endpoint.Binding.Cookie)
        END IF
        IF Endpoint.Binding.Request.Headers.getLength() > 0 THEN
            FOR ind = 1 TO Endpoint.Binding.Request.Headers.getLength()
                CALL req.setHeader(
                    Endpoint.Binding.Request.Headers[ind].Name,
                    Endpoint.Binding.Request.Headers[ind].Value)
            END FOR
        END IF
        CALL Endpoint.Binding.Response.Headers.clear()
        IF Endpoint.Binding.ConnectionTimeout <> 0 THEN
            CALL req.setConnectionTimeOut(Endpoint.Binding.ConnectionTimeout)
        END IF
        IF Endpoint.Binding.ReadWriteTimeout <> 0 THEN
            CALL req.setTimeOut(Endpoint.Binding.ReadWriteTimeout)
        END IF
        IF Endpoint.Binding.CompressRequest IS NOT NULL THEN
            CALL req.setHeader(
                "Content-Encoding", Endpoint.Binding.CompressRequest)
        END IF

        # Perform request
        CALL req.setMethod("GET")
        CALL req.setHeader("Accept", "text/plain")
        CALL req.doRequest()

        # Retrieve response
        LET resp = req.getResponse()
        # Process response
        INITIALIZE resp_body TO NULL
        LET contentType = resp.getHeader("Content-Type")
        IF resp.getHeaderCount() > 0 THEN
            # Retrieve response runtime headers
            FOR ind = 1 TO resp.getHeaderCount()
                LET headerName = resp.getHeaderName(ind)
                CALL Endpoint.Binding.Response.Headers.appendElement()
                LET Endpoint.Binding.Response.Headers[
                        Endpoint.Binding.Response.Headers.getLength()].Name =
                    headerName
                LET Endpoint.Binding.Response.Headers[
                        Endpoint.Binding.Response.Headers.getLength()].Value =
                    resp.getHeader(headerName)
            END FOR
        END IF
        CASE resp.getStatusCode()

            WHEN 200 #Success
                IF contentType MATCHES "*text/plain*" THEN
                    # Parse TEXT response
                    LET txt = resp.getTextResponse()
                    LET resp_body = txt
                    RETURN C_SUCCESS, resp_body
                END IF
                LET wsError.code = resp.getStatusCode()
                LET wsError.description = "Unexpected Content-Type"
                RETURN -1, resp_body

            OTHERWISE
                LET wsError.code = resp.getStatusCode()
                LET wsError.description = resp.getStatusDescription()
                RETURN -1, resp_body
        END CASE
    CATCH
        LET wsError.code = status
        LET wsError.description = sqlca.sqlerrm
        RETURN -1, resp_body
    END TRY
END FUNCTION

In a README on the server side you will find the curl example that can be used to call each of the sample web services.

curl "http://localhost:8091/ws/r/Calculator/Add_wsquery?x=1&y=2"

Through looking through the various examples, it is a case of recognising that various curl arguments correlate to 4gl methods.

For example using -X POST corresponds to CALL req.setMethod("POST").  Passing headers using -H "header_name: header_value" corresponds to CALL req.setHeader(header_name, header_value) etc

You might then say that there is a lot of code in this generated client code, certainly compared to the simple examples in the documentation.   That is true but if you look into it, the code referencing the EndPoint allows you to overwrite some of the parameters in the generated code such as the Uri, add additional cookies and headers, and to set the timeouts.    Also when checking the response, the code handles any errors.  Code that is not in the simple examples, and code that youc an omit when prototyping.  If you do have to write the equivalent of this generated code, you can refer to the generated code as a good template.

In summary if you do find yourself having to interact with a web service that does not provide an OpenAPI definition, look in the API documentation for a curl example or similar, and ask yourself how do I map each of these different curl arguments to com.HttpRequest methods.  Build yourself a little mapping table and then code a 4gl module that is similar to what might have been generated.