Tuesday, January 13, 2015

Calling Stored Procedures From HTML

Question: How do I call a SQL Anywhere stored procedure from HTML without refreshing the whole web page?

Answer: You can use the XMLHttpRequest JavaScript object to send and receive data to and from a SQL Anywhere web service that calls the stored procedure.

Here is what W3Schools has to say:


The XMLHttpRequest object is a developer's dream, because you can:
  • Update a web page without reloading the page

  • Request data from a server after the page has loaded

  • Receive data from a server after the page has loaded

  • Send data to a server in the background

If you've read about XMLHttpRequest in the past and been scared off, here's what you don't have to deal with using it:
  • XML: In spite of the name, XMLHttpRequest can return ordinary string data.

  • AJAX: Your stored procedure calls can be synchronous (call and return) rather than asynchronous (fire and forget).

  • Frameworks: The XMLHttpRequest object is useful all by itself, you don't have to commit to a massive library.

  • Key-value stores, NoSQL, JQuery, Xpath, JSON... the list of stuff you don't need goes on and on.
The Mozilla Developer Network has some of the best docs for XMLHttpRequest.

The code below shows two pairs of like-named web services and stored procedures:
  • "display" returns a static web page with a search-as-you-type input field that retrieves rows from the Customer table in the SQL Anywhere 16 demo database.

  • "search" is called via XMLHttpRequest from the display web page; it receives a search string as a parameter and returns a string of HTML.
The following URLs are used to launch the two web services:
http://localhost:12345/display     - from the web browser
search?searchString=xx...          - from JavaScript via XMLHttpRequest
Here's what the display web service and procedure look like:
CREATE SERVICE display
   TYPE 'RAW' AUTHORIZATION OFF USER DBA
   AS CALL display();

CREATE PROCEDURE display()
RESULT ( html_string LONG VARCHAR )
BEGIN
CALL dbo.sa_set_http_header( 'Content-Type', 'text/html' );
SELECT STRING ( 
   '<HTML> ',
   '<HEAD> ',
   '<STYLE> ',
      'TABLE { padding: 0; border: 1px solid black; border-collapse: collapse; } ', 
      'TD { padding: 0.3em; border: 1px solid black; } ', 
   '</STYLE> ',
   '<SCRIPT> ',
   'function showResults ( searchString ) { ',
      'xmlHttp = new XMLHttpRequest(); ',
      'xmlHttp.open ( "GET", "search?searchString=" + encodeURI ( searchString ), false ); ',
      'xmlHttp.send(); ',
      'document.getElementById ( "searchResults" ).innerHTML = xmlHttp.responseText; ',
   '} ',
   '</SCRIPT> ', 
   '</HEAD> ',
   '<BODY> ',
   '<INPUT TYPE="text" ID="txt1" onkeyup="showResults ( this.value )" /> ',
   '<P> ',
   '<SPAN ID="searchResults"></SPAN> ',
   '</BODY> ',
   '</HTML> ' );
END;
The CREATE SERVICE on lines 1 to 3 sets up a web service wrapper for the display procedure.

The CREATE PROCEDURE on lines 5 to 31 returns a static web page that contains the input field and the XMLHttpRequest logic.
Note: The display web page could be stored in a static HTML text file instead of a web service if it wasn't for the problem of not being not being able to perform cross-domain requests from a file:
XMLHttpRequest cannot load file: ... Cross origin requests are only supported for protocol schemes: http, data, chrome-extension, https, chrome-extension-resource.
To avoid that problem, this web page could also be served up by the general-purpose "web service for website files" described in the earlier article Embedding Fiori In SQL Anywhere.
The HTML INPUT tag on line 26 calls the local JavaScript showResults function whenever a character is typed or deleted.

The JavaScript showResults function on lines 17 through 22 does the following:
  • A new instance of the JavaScript XMLHttpRequest object is created on line 18.

  • The XMLHttpRequest.open method is called on line 19 to set up an HTTP GET operation that passes the searchString value to the search service.

  • The JavaScript encodeURI function works like the SQL HTTP_ENCODE function to ensure that special characters like spaces are not lost when passed in URLs.

  • The third argument on line 19 turns off the asynchronous behavior of XMLHttpRequest so the send() on line 20 works as a call-and-return rather than fire-and-forget.

  • The assignment statement on line 21 displays the HTML returned via XMLHttpRequest

OK, it's not exactly "Calling A Stored Procedure From HTML", it's calling a stored procedure from JavaScript, but JavaScript is a fact of life when developing HTML web pages... JavaScript is arguably the most important programming language in the world today.

Here's what the search web service and procedure look like:
CREATE SERVICE search 
   TYPE 'RAW' AUTHORIZATION OFF USER DBA
   AS CALL search ( :searchString );

CREATE PROCEDURE search ( IN @searchString VARCHAR ( 100 ) )
RESULT ( html_string LONG VARCHAR )
BEGIN
CALL dbo.sa_set_http_header( 'Content-Type', 'text/html' );
SELECT STRING ( 
          '<TABLE>', 
          LIST ( STRING ( 
             '<TR>',
             '<TD>', ID, '</TD>',
             '<TD>', Surname, '</TD>',
             '<TD>', GivenName, '</TD>',
             '<TD>', Street, '</TD>',
             '<TD>', City, '</TD>',
             '<TD>', State, '</TD>',
             '<TD>', Country, '</TD>',
             '<TD>', PostalCode, '</TD>',
             '<TD>', Phone, '</TD>',
             '<TD>', CompanyName, '</TD>',
             '</TR>\X0D\X0A' ),
             ''
             ORDER BY ID ),
          '</TABLE>' )
  FROM Customers
 WHERE Surname                    LIKE STRING ( @searchString , '%' )
    OR GivenName                  LIKE STRING ( @searchString , '%' )
    OR Street                     LIKE STRING ( '%', @searchString , '%' )
    OR City                       LIKE STRING ( @searchString , '%' )
    OR STRING ( State, '=state' ) = @searchString 
    OR PostalCode                 LIKE STRING ( @searchString , '%' )
    OR Phone                      LIKE STRING ( @searchString , '%' )
    OR CompanyName                LIKE STRING ( '%', @searchString , '%' );
END;
The SELECT on lines 9 through 35 builds an HTML TABLE containing all the columns in the Customer table.

The WHERE clause on lines 28 through 35 applies the search string to eight of those columns, with a few cute twists:
  • The search string is matched against leading characters in Surname, GivenName, City, PostalCode and Phone.

  • The search string is matched against any substring in Street and CompanyName.

  • The special format 'xx=state' lets the user specify exact State values.
Those cute twists aren't important in themselves, they only serve to illustrate that all the power of SQL queries is available when calling stored procedures from JavaScript.

Here's are the Windows command line for starting the SQL Anywhere 16 demo database with the builtin HTTP server enabled on port 12345, and then launching an ISQL session so you can load the code shown above:
"%SQLANY16%\bin64\dbspawn.exe"^
  -f "%SQLANY16%\bin64\dbsrv16.exe"^
  -o dbsrv16_log_demo.txt^
  -x tcpip^
  -xs http(port=12345;maxsize=0;to=600;kto=600)^
  "C:\Users\Public\Documents\SQL Anywhere 16\Samples\demo.db"

"%SQLANY16%\bin64\dbisql.com"^
  -c "ENG=demo; DBN=demo; UID=dba; PWD=sql; CON=demo-1"
The following screen capture doesn't do justice to the search-as-you-type action, you really have to try it yourself:


Thursday, January 1, 2015

Embedding Fiori In SQL Anywhere


Update January 7, 2015: This article has been updated to reflect Mark Culp's comment about using LONG BINARY and the CSCONVERT() function, plus the new website_file.csconvert_required column.
Fiori is the name given to a new style of user interface developed using the SAPUI5 client-side HTML5 and JavaScript library. This article shows how to embed the SAPUI5 library in a SQL Anywhere database so you can build Fiori-style applications with the following advantages:
  • All the scripts and data are stored in a database where they are protected from loss, attack and accidental modification.

  • All the scripts and data are available locally when the internet connection is lost.
In particular, this article shows how to:
  1. Store text and binary files in a SQL Anywhere table so they can be served up as HTTP responses by SQL Anywhere's built-in web server.

  2. Code a SQL Anywhere "root" web service that returns rows from the table just like an ordinary web server returns files from the server.

  3. Use a simple HTML page to show that the table and web service work as advertised.

  4. Download all the SAPUI5 Fiori files into your SQL Anywhere database so they'll work just like the files stored on the internet.

  5. Show how the "SAPUI5 in 20 Seconds" Fiori demo can be made to work from inside SQL Anywhere by making a one-line change.

1. Store Website Files In SQL Anywhere

Here's a Windows batch file that creates and starts a new SQL Anywhere database and then launches an ISQL session for running SQL commands:
"%SQLANY16%\bin64\dbinit.exe" website.db

"%SQLANY16%\bin64\dbspawn.exe"^
  -f "%SQLANY16%\bin64\dbsrv16.exe"^
  -o dbsrv16_log_website.txt^
  -x tcpip^
  -xs http(port=12345;maxsize=0;to=600;kto=600)^
  website.db 

"%SQLANY16%\bin64\dbisql.com"^
  -c "ENG=website; DBN=website; UID=dba; PWD=sql; CON=website-1"
The dbsrv16.exe -xs option specifies port 12345 for HTTP traffic so SQL Anywhere doesn't compete with other processes that might already be using port 80 (such as Foxhound or Skype).

Here's the table that can hold all the HTML, CSS, JavaScript and other files that make up an file-based website:
CREATE TABLE website_file (
   file_name                   VARCHAR ( 260 ) NOT NULL PRIMARY KEY,
   content_type                VARCHAR ( 100 ) NOT NULL,
   csconvert_required          VARCHAR ( 1 ) NOT NULL,
   contents                    LONG BINARY NOT NULL );
The file_name column holds the relative URL for each file, and the data is stored in the contents column.

The content_type and csconvert_required columns are filled in automatically when a row is inserted or updated:
CREATE TRIGGER insert_update_website_file
   BEFORE INSERT, UPDATE ON website_file 
   REFERENCING NEW AS new_website_file
   FOR EACH ROW
BEGIN
DECLARE @extension_pos INTEGER;
DECLARE @extension     VARCHAR ( 100 );

SET new_website_file.file_name = TRIM ( REPLACE ( new_website_file.file_name, '\\', '/' ) );

SET @extension_pos = LOCATE ( new_website_file.file_name, '.', -1 );

SET @extension 
   = IF @extension_pos = 0
        THEN ''
        ELSE SUBSTR ( new_website_file.file_name, @extension_pos + 1 )
     ENDIF;

CASE @extension
   WHEN 'bin'    THEN 
      SET new_website_file.content_type       = 'application/octet-stream';
      SET new_website_file.csconvert_required = 'N';
   WHEN 'css'    THEN 
      SET new_website_file.content_type       = 'text/css';
      SET new_website_file.csconvert_required = 'Y';
   WHEN 'dll'    THEN 
      SET new_website_file.content_type       = 'application/octet-stream';
      SET new_website_file.csconvert_required = 'N';
   WHEN 'doc'    THEN 
      SET new_website_file.content_type       = 'application/msword';
      SET new_website_file.csconvert_required = 'N';
   WHEN 'exe'    THEN 
      SET new_website_file.content_type       = 'application/octet-stream';
      SET new_website_file.csconvert_required = 'N';
   WHEN 'gif'    THEN 
      SET new_website_file.content_type       = 'image/gif';
      SET new_website_file.csconvert_required = 'N';
   WHEN 'htm'    THEN 
      SET new_website_file.content_type       = 'text/html';
      SET new_website_file.csconvert_required = 'Y';
   WHEN 'html'   THEN 
      SET new_website_file.content_type       = 'text/html';
      SET new_website_file.csconvert_required = 'Y';
   WHEN 'jpg'    THEN 
      SET new_website_file.content_type       = 'image/jpeg';
      SET new_website_file.csconvert_required = 'N';
   WHEN 'jpeg'   THEN 
      SET new_website_file.content_type       = 'image/jpeg';
      SET new_website_file.csconvert_required = 'N';
   WHEN 'js'     THEN 
      SET new_website_file.content_type       = 'application/javascript';
      SET new_website_file.csconvert_required = 'Y';
   WHEN 'json'   THEN 
      SET new_website_file.content_type       = 'application/json';
      SET new_website_file.csconvert_required = 'Y';
   WHEN 'pdf'    THEN 
      SET new_website_file.content_type       = 'application/pdf';
      SET new_website_file.csconvert_required = 'N';
   WHEN 'png'    THEN 
      SET new_website_file.content_type       = 'image/png';
      SET new_website_file.csconvert_required = 'N';
   WHEN 'saplan' THEN 
      SET new_website_file.content_type       = 'application/xml';
      SET new_website_file.csconvert_required = 'Y';
   WHEN 'txt'    THEN 
      SET new_website_file.content_type       = 'text/plain';
      SET new_website_file.csconvert_required = 'Y';
   WHEN 'xml'    THEN 
      SET new_website_file.content_type       = 'application/xml';
      SET new_website_file.csconvert_required = 'Y';
   WHEN 'zip'    THEN 
      SET new_website_file.content_type       = 'application/zip';
      SET new_website_file.csconvert_required = 'N';
   ELSE               
      SET new_website_file.content_type       = 'text/html';
      SET new_website_file.csconvert_required = 'Y';
END CASE;

END;
The SET on line 9 changes all the Windows "\" characters to URL-friendly forward slashes.

The code on lines 11 through 77 extracts the extension (e.g., .html) from the file_name value, then uses it to fill in the content_type and cs_convert_required columns.

2. Create A Web Service For Website Files

This web service is like no other: it has no name, and no parameters (well, it does have the name "root" but that's syntactic sugar meaning "no name").

This is the web service that gets called when no other web service matches the URL, and that's what happens when an URL specifies one of the file_name values from the website_file table; e.g., http://localhost:12345/Hello_World.html
CREATE SERVICE root 
   TYPE 'RAW' AUTHORIZATION OFF URL ON USER DBA
   AS CALL root();

CREATE PROCEDURE root() RESULT ( html_string LONG BINARY )
BEGIN
   
DECLARE @url                LONG VARCHAR;
DECLARE @content_type       LONG VARCHAR;
DECLARE @csconvert_required VARCHAR ( 1 );

SET @url = HTTP_VARIABLE ( 'URL' );

IF EXISTS ( SELECT * 
              FROM website_file 
             WHERE website_file.file_name = @url ) THEN

   SELECT website_file.content_type,  
          website_file.csconvert_required 
     INTO @content_type,
          @csconvert_required
     FROM website_file
    WHERE website_file.file_name = @url;

   CALL dbo.sa_set_http_header( 'Content-Type', @content_type );

   SELECT IF @csconvert_required = 'Y'
             THEN CSCONVERT ( website_file.contents, CONNECTION_PROPERTY ( 'CharSet' ) )
             ELSE website_file.contents  
          END IF
     FROM website_file
    WHERE website_file.file_name = @url;

ELSE

   CALL dbo.sa_set_http_header( 'Content-Type', 'text/html' );

   SELECT STRING ( 'URL not found: "', @url, '"' );

END IF;

END;
The CREATE SERVICE statement on lines 1 to 3 defines a service-with-no-name that calls the web_server procedure with no parameters.

The SET statement on line 12 copies the URL value from the HTTP header of the same name.

The SELECT on lines 18 through 23 determines what 'Content-Type' value to use in the CALL on line 25, and whether or not the CSCONVERT() function should be called from the IF expression on lines 27 through 30.

The SELECT on lines 27 through 32 returns the website_file.contents to the browser.

3. Run Hello_World.html

Here's what the Hello_World.html file looks like:
<HTML>
<TITLE>Hello World!</TITLE>
<BODY>
Hello World!
</BODY>
</HTML>
Here's how to load the file into the table; note that the trigger takes care of the content-type column:
INSERT website_file ( file_name, contents ) 
   ON EXISTING UPDATE 
   SELECT 'Hello_World.html', xp_read_file ( 'Hello_World.html' );
COMMIT;
Here's what SQL Anywhere shows in the browser when you specify http://localhost:12345/Hello_World.html:


4. Download SAPUI5 Into SQL Anywhere

The SAPUI5 files can be downloaded from Open UI5.

For the purposes of this article, the SAPUI5 files were downloaded to C:/download/Open UI5 Runtime 1.24.3.

The 4,300 files in the resources subfolder comprise the SAPUI5 library; here's what they look like:

The following code shows how a CLASS 'DIRECTORY' proxy table was used to load the SAPUI5 library:
CREATE OR REPLACE VARIABLE @root LONG VARCHAR = 'C:/download/Open UI5 Runtime 1.24.3/resources/';

CREATE SERVER folder
   CLASS 'DIRECTORY' USING 'ROOT={@root};SUBDIRS=10';

CREATE EXTERNLOGIN DBA TO folder;

CREATE EXISTING TABLE proxy_folder AT 'folder;;;.';

INSERT website_file (
       file_name,
       contents )
SELECT proxy_folder.file_name,
       proxy_folder.contents
  FROM proxy_folder
 WHERE LEFT ( proxy_folder.permissions, 1 ) <> 'd'
   AND proxy_folder.size > 0
 ORDER BY proxy_folder.file_name;
COMMIT;
The code on lines 1 through 8 uses the techniques shown in Proxy {Variables} to create a proxy table that represents the resources folder and all its subfolders, and the INSERT SELECT on lines 10 through 18 loads all the files (just the files, not the 'd' directories) into the website_file table.

Here's what the table looked like after the INSERT SELECT;
SELECT website_file.file_name
  FROM website_file;

file_name
'Hello_World.html'
'.theming'
'jquery-sap-dbg.js'
'jquery-sap.js'
'jquery.sap.act-dbg.js'
...
'sap/m/.theming'
'sap/m/ActionListItem-dbg.js'
'sap/m/ActionListItem.js'
'sap/m/ActionListItemRenderer-dbg.js'
'sap/m/ActionListItemRenderer.js'
...
Note that the file_name field contains the relative path subfolder names beneath the resources folder so that different files with the same name located in different subfolders can be differentiated from one another.

5. Run "SAPUI5 in 20 Seconds" In SQL Anywhere


The original "SAPUI5 in 20 Seconds" demo can be found here: How to Create and Run a Simple SAPUI5 Application from Scratch within 20 Seconds.

Here's what it looks like after this single line
src="https://sapui5.hana.ondemand.com/resources/sap-ui-core.js"
was changed to this
src="http://localhost:12345/sap-ui-core.js"
and stored in the file SAPUI5_SQL_Anywhere_in_20_Seconds.html:
<!DOCTYPE html>
<html>
<head>
    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
    <meta http-equiv="Content-Type" content="text/html;charset=UTF-8"/>
    <title>SAPUI5 in 20 Seconds</title>

    <!-- 1.) Load SAPUI5 (from a remote server), select theme and control library -->
    <script id="sap-ui-bootstrap"
        src="http://localhost:12345/sap-ui-core.js"
        data-sap-ui-theme="sap_goldreflection"
        data-sap-ui-libs="sap.ui.commons"></script>
    <!-- 2.) Create a UI5 button and place it onto the page -->
    <script>
        // create the button instance
        var myButton = new sap.ui.commons.Button("btn");
        // set properties, e.g. the text (there is also a shorter way of setting several properties)
        myButton.setText("Hello World!");
       // attach an action to the button's "press" event (use jQuery to fade out the button)
        myButton.attachPress(function(){$("#btn").fadeOut()});
        // place the button into the HTML element defined below
        myButton.placeAt("uiArea");
        // an alternative, more jQuery-like notation for the same is:  
        /*
        $(function(){
            $("#uiArea").sapui("Button", "btn", {
                text:"Hello World!",
                press:function(){$("#btn").fadeOut();}
            });
         });
         */
     </script>
</head>
<body class="sapUiBody">

    <!-- This is where you place the UI5 button -->
    <div id="uiArea"></div>
</body>
</html>
Here's how to load the modified file into the SQL Anywhere table:
INSERT website_file ( file_name, contents ) 
ON EXISTING UPDATE 
SELECT 'SAPUI5_SQL_Anywhere_in_20_Seconds.html', 
       xp_read_file ( 'SAPUI5_SQL_Anywhere_in_20_Seconds.html' );
COMMIT;
Here's the URL to launch the modified "SAPUI5 in 20 Seconds" demo:
http://localhost:12345/SAPUI5_SQL_Anywhere_in_20_Seconds.html
Note that the button shows the Fiori-style color change when the cursor hovers over it:


Tip: When something goes wrong, use the browser's built-in debugger. Chrome is best; the latest version of Firefox is pretty good, but IE's debugger sucks.

For example, if you try to run the "SAPUI5 in 20 Seconds" demo from the file stored on your hard drive instead of the one in the database, like this
file:///C:/Temp/SAPUI5_SQL_Anywhere_in_20_Seconds.html
you will see absolutely nothing in the browser window. Click right mouse - Inspect element to open the debugger to see the dreaded "not allowed access" error:
XMLHttpRequest cannot load
No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'null' is therefore not allowed access.
Here's what it looks like in the Chrome debugger:
To find a solution just copy and paste the error message text into Google; chances are, many others have found and fixed and documented the same problem (in this case, there are workarounds that will let you mix HTML files on the local disk and in the database, but the simplest solution is to load everything into the database so it all has the same "origin".)

Oh, and if you see several messages, just make one fix and test again... chances are that all the messages have the same cause.

And don't feel bad if you don't completely understand everything the debugger is showing you; you're not looking for a deep understanding of the internet, just a fix for your application.