Friday, April 12, 2013

Logging Server Messages

SQL Anywhere engines display startup, progress and diagnostic text messages in the server window. Historically, the server window was called the "database console" so the messages are sometimes called the "console log".



The server messages may also include custom debugging and other output written by MESSAGE TO CONSOLE statements, and the command line option -o filespec.txt can be specified on the server command line to save all the messages to a text file:

"%SQLANY16%\bin64\dbsrv16.exe"^
  -o dbsrv16_log_ddd16.txt^
  ddd16.db 

MESSAGE 'Hello, world!' TO CONSOLE;

I. 04/08 09:07:11. SQL Anywhere Network Server Version 16.0.0.1324 I. 04/08 09:07:11. Developer edition, not licensed for deployment.
I. 04/08 09:07:11.  I. 04/08 09:07:11. Copyright © 2013 SAP AG or an SAP affiliate company.
...
I. 04/08 09:07:11. Automatic tuning of multiprogramming level is enabled
I. 04/08 09:07:11. Starting database "ddd16" (C:\temp\ddd16.db) at Mon Apr 08 2013 09:07
I. 04/08 09:07:11. Performance warning: Database file "C:\temp\ddd16.db" consists of 15 disk fragments
I. 04/08 09:07:11. Transaction log: ddd16.log
I. 04/08 09:07:11. Starting checkpoint of "ddd16" (ddd16.db) at Mon Apr 08 2013 09:07
I. 04/08 09:07:12. Finished checkpoint of "ddd16" (ddd16.db) at Mon Apr 08 2013 09:07
I. 04/08 09:07:12. Database "ddd16" (ddd16.db) started at Mon Apr 08 2013 09:07
I. 04/08 09:07:12. Database server started at Mon Apr 08 2013 09:07
I. 04/08 09:07:12. Trying to start SharedMemory link ...
I. 04/08 09:07:12.     SharedMemory link started successfully
I. 04/08 09:07:12. Trying to start TCPIP link ...
I. 04/08 09:07:12. Starting on port 2638
I. 04/08 09:07:17.     TCPIP link started successfully
I. 04/08 09:07:17. Now accepting requests
I. 04/08 09:08:30. Hello, world!
If you want to preserve a permanent record of these server messages, it is possible to gather them up at runtime and save them in the database. Unlike external files which can get lost and confused over time, data in the database is protected from inconsistencies by the ACID rules for database transactions, as well as being protected from loss by the regular database backup process (you do have a regular database backup process, right? ...riiight? :)

The sa_server_messages() system procedure makes the "gather them up at runtime" process possible, and some custom SQL takes care of the "save them in the database" part.

Here's a table designed to save all the msg_* columns returned by calls to sa_server_messages():
CREATE TABLE server_messages (
   primary_key    BIGINT NOT NULL DEFAULT AUTOINCREMENT PRIMARY KEY,
   msg_id         UNSIGNED BIGINT NOT NULL,
   msg_text       LONG VARCHAR NOT NULL,
   msg_time       TIMESTAMP NOT NULL,
   msg_severity   VARCHAR ( 255 ) NOT NULL,
   msg_category   VARCHAR ( 255 ) NOT NULL,
   msg_database   VARCHAR ( 255 ) NOT NULL,
   UNIQUE ( msg_id, msg_time ) );
The primary_key column has been added to guarantee ordering in the face of these facts:
  • msg_id starts over at zero every time the server is restarted,

  • it is conceivable that msg_time may repeat for rapid-fire messages, and

  • msg_time values may fall back by one hour (and thus overlap if not repeat) when the autumn Daylight Saving Time change occurs.
At this point a bigger question arises: "If sa_server_messages() has access to all the server messages, why bother to save them separately? Why not just call sa_server_messages() whenever you want to query the messages?"

The answer is, only recent server messages are available to sa_server_messages(), so they must be repeatedly retrieved and saved in a separate table if you want to keep them longer. It is possible to define what "recent" means by setting the MessageCategoryLimit option but no matter what value you pick it may be possible for a flood of message output to cause data loss before you get around to calling sa_server_messages().

There are other advantages to using a permanent table:
  • It's easier to replicate server messages to the consolidated database using SQL Remote or MobiLink if an ordinary table is involved, and

  • different rules can be used for purging old data; e.g., delete by msg_time, delete by msg_category, delete uninteresting messages, and so on.
But the big reason is this: If the server's restarted, sa_server_messages() starts over from scratch... all the old messages are gone, gone, gone.

The following code uses a SQL Anywhere EVENT triggered once a minute to capture fresh messages. The code ignores the problem of inter-call message loss by assuming you can either increase the MessageCategoryLimit option and/or change the EVERY 1 MINUTES clause to something more frequent.
CREATE EVENT save_server_messages 
SCHEDULE START TIME '00:00' EVERY 1 MINUTES
HANDLER BEGIN

DECLARE @most_recent_msg_time   TIMESTAMP DEFAULT '1900-01-01';

-- Determine the approximate starting point for fresh messages.

SELECT TOP 1 msg_time 
  INTO @most_recent_msg_time
  FROM server_messages
 ORDER BY primary_key DESC;

-- Copy fresh messages.

INSERT server_messages (
       msg_id,
       msg_text,
       msg_time,
       msg_severity,
       msg_category,
       msg_database )
SELECT msg_id, 
       COALESCE ( msg_text, '' ),
       msg_time,
       msg_severity,
       msg_category,
       COALESCE ( msg_database, '' )
  FROM sa_server_messages()
 WHERE msg_time >= DATEADD ( hour, - 1, @most_recent_msg_time )
   AND NOT EXISTS  
          ( SELECT *
              FROM server_messages
             WHERE server_messages.msg_id   = sa_server_messages.msg_id
               AND server_messages.msg_time = sa_server_messages.msg_time ) 
 ORDER BY msg_id;

COMMIT;

END;
  • The SELECT starting on line 9 determines an approximate starting point for copying fresh messages. It's an approximate value because Daylight Saving Time causes the clock-on-the-wall time to be set earlier by one hour each autumn, and messages recorded during that hour must be treated as fresh messages even though they may have msg_time values earlier than @most_recent_msg_time.

  • The INSERT SELECT starting on line 16 copies and saves all the fresh messages returned by sa_server_messages().

  • The COALESCE calls make life easier later on: no worries about NULL values when writing queries.

  • The WHERE msg_time >= DATEADD predicate eliminates messages older than 1 hour.

  • The NOT EXISTS predicate eliminates more recent messages that have already been copied. ON EXISTING SKIP is often a good alternative to this kind of NOT EXISTS, but not this time: ON EXISTING SKIP allows new DEFAULT AUTOINCREMENT values to be calculated and then discarded when the primary key value collides; it also requires a different primary key for the server_messages table... something other than a DEFAULT AUTOINCREMENT column (yes, this has been tested, and yes, ON EXISTING SKIP does work as long as you're OK with big gaps in the DEFAULT AUTOINCREMENT values).

  • The ORDER BY msg_id clause makes sure that new DEFAULT AUTOINCREMENT values of the server_message.primary_key column are calculated in the same order as the messages were created. By definition, all rows returned by a single call to sa_server_messages() have monotonically increasing values of msg_id even though msg_id starts over again when the server is restarted. Put another way, sa_server_messages.msg_id works as a primary key for all the messages produced by a single execution of the SQL Anywhere server, which is all that sa_server_messages() returns.
Flaw: It is conceivable that the SQL Anywhere server could be shut down and restarted around the time the clock is set backward for Daylight Saving Time, causing the same values of msg_id and msg_time to be used for two different messages, and that the WHERE clause could cause one of those messages to be incorrectly discarded. The solution is left as an exercise for the person who cares reader :)

Here's what the table looks like...




Wednesday, April 10, 2013

The Newsgroups Are Gone

The old NNTP newsgroups for SQL Anywhere and other Sybase products have been shut down... they are gone, kaput, no more, expired, extinct.



The old content has been moved to a read-only website; if you go there, then drill down to SQL Anywhere, then drill down again (for example) to General Discussion, you get here:



Yup... all thirty-three thousand questions and one hundred thousand responses... page one, page two, page three hundred and thirty eight.

But hey! You can "Sort by: Response Posting Date", whee! That moves... some stuff... around a little bit.

No "Search" field though. And if you want to look for something in more than one SQL Anywhere forum, you have to scroll through... each... list... separately.

Google Search does work, but as far as Google is concerned, http://nntp-archive.sybase.com is one single repository. That's different from the old Google Groups where single newsgroups and groups of like-named newsgroups could be searched apart from all the others.

For example, if you do a Google Search for "insert" on the new archive website

"insert" site:http://nntp-archive.sybase.com
you get 9950 hits, whereas if you search the SQL Anywhere newsgroups via Google Search on Google Groups,
"insert" group:sybase.public.sqlanywhere.*
you get a more specific list of 6430 hits. The difference is that ASE and other newsgroups are included in the first result set but not the second.

The problem gets worse if you search on terms that aren't database-specific in themselves. For example, if you want to see all the discussions involving Java or PowerBuilder applications that are using SQL Anywhere databases, here's what Google Search returns:
Google Search                                          Hits
--------------------------------------------------     ----

"java" site:http://nntp-archive.sybase.com            36100
"java" group:sybase.public.sqlanywhere.*               3650

"powerbuilder" site:http://nntp-archive.sybase.com   107000
"powerbuilder" group:sybase.public.sqlanywhere.*       3360
Sadly, even Google Groups is not a reliable source for newsgroup searches, but it is certainly better than the new website... and now that the underlying Sybase NNTP server is gone the data may also disappear from Google Groups.

Once upon a time, promises were made to import old NNTP discussions into the new SQL Anywhere Forum. Those promises have not yet been fulfilled, but there's still hope.



Correction: It IS possible to narrow the focus of Google searches on the new website.

Rather than trying to specify the newsgroup name in the "site:" parameter, simply add part or all of the newsgroup name as an ordinary search term.

For example, add sybase.public.sqlanywhere.general, or sybase.public.sqlanywhere for all the SQL Anywhere newsgroups.
Google Search                                                                           Hits
--------------------------------------------------                                      ----

"java" site:http://nntp-archive.sybase.com                                              36200
"java" sybase.public.sqlanywhere site:http://nntp-archive.sybase.com                    17900
"java" sybase.public.sqlanywhere.general site:http://nntp-archive.sybase.com            10900

"powerbuilder" site:http://nntp-archive.sybase.com                                     103000
"powerbuilder" sybase.public.sqlanywhere site:http://nntp-archive.sybase.com            23700
"powerbuilder" sybase.public.sqlanywhere.general site:http://nntp-archive.sybase.com    12100
Clearly, this shows that the new website is better than Google Groups because it contains more data as well as allowing focussed searches... thanks to Jason Hinsperger for pointing this out.


Monday, April 8, 2013

Latest SQL Anywhere EBFs and Docs

Current builds for the active platforms...

HP-UX     12.0.1.3798 EBF           29 Oct 2012
 Itanium  11.0.1.2879 EBF           31 Oct 2012

IBM AIX   12.0.1.3798 EBF           24 Oct 2012
          11.0.1.2879 EBF           29 Oct 2012

Linux     16.0.0.1324 GA            05 Mar 2013
          12.0.1.3873 EBF       *** 05 Apr 2013 ***
          11.0.1.2913 EBF           19 Dec 2012

Mac OS    12.0.1.3871 EBF       *** 05 Apr 2013 ***
          11.0.1.2449 EBF           29 Jun 2010

Solaris   12.0.1.3798 EBF           24 Oct 2012
 SPARC    11.0.1.2913 EBF (SA)      19 Dec 2012

Solaris   12.0.1.3798 EBF           29 Oct 2012
 x64      11.0.1.2879 EBF           29 Oct 2012

Windows   16.0.0.1324 GA            05 Mar 2013
          12.0.1.3867 EBF           22 Mar 2013
          12.0.1 French Docs,       25 Sep 2012
                 English Docs,      25 Sep 2012
                 German Docs,       25 Sep 2012
                 Chinese Docs,  *** 28 Mar 2013 ***
                 Japanese Docs  *** 28 Mar 2013 ***
          11.0.1.2913 EBF           21 Dec 2012

Other Stuff...

 Older EBFs

 Free support! Q&A forum
   ...or, call Tech Support

 SQL Anywhere...
   ...Sybase home page 
   ...SAP home page 
   ...SAP Developer Center 

 Buy SQL Anywhere 

 Developer Edition... 
   [16.0] [12.0.1] [11.0.1]

 Download the...
   Educational Edition 
   Web Edition 

 Supported Platforms...
   SQL Anywhere 
   Linux 
   OnDemand

 ODBC Drivers for MobiLink

The three asterisks "***" show which Express Bug Fixes (EBFs) and GA builds have appeared on the website since the previous version of this page.
  • Only the latest fully-supported versions of SQL Anywhere (11.0.1, 12.0.1 and 16.0) are shown here.

  • Just because an older version or different platform isn't "fully supported" any more doesn't mean you can't download files (or ask questions, or get help), it just means there won't be any more new EBFs released.


Tuesday, April 2, 2013

It's just an expression

Rather than use a formal notation to present the SQL syntax used in SQL Anywhere, the Help has always used an informal, flexible "user-friendly" notation.

Sadly, with informality comes imprecision, which leads to confusion, in one case involving the syntax for "expression":

Expressions

An expression is a statement that can be evaluated to return values.
expression:
   case-expression
  | constant
  | [correlation-name.]column-name
  | - expression
  | expression operator expression
  | ( expression )
  | function-name ( expression, ... )
  | if-expression
  | special value
  | ( subquery )
  | variable-name
  | sequence-expression
The problem is, SQL Anywhere implements more than one form of expression depending on the context, and the Help only describes one form. In particular, the definition of "expression" includes "( subquery )", and the syntax for the CALL statement allows for expressions as parameters, leading one to believe that subqueries can be passed as CALL parameters:
CALL statement

Invokes a procedure.
[variable = ] CALL procedure-name ( [ expression, ... ] )
Unfortunately, in SQL Anywhere 12 and earlier, such was not the case:
CALL sa_db_info ( ( SELECT 0 FROM DUMMY ) );

Could not execute statement.
Syntax error near ')' on line 1
SQLCODE=-131, ODBC 3 State="42000"
Line 1, column 1
The good news, SQL Anywhere 16 has caught up with the Help in this respect; subqueries are now allowed in CALL statements:
CALL sa_db_info ( ( SELECT 0 FROM DUMMY ) );

Number,Alias,File,ConnCount,PageSize,LogName
0,'ddd16','C:\\projects\\$SA_templates\\ddd16.db',2,4096,'C:\\projects\\$SA_templates\\ddd16.log'



Thursday, March 28, 2013

Latest SQL Anywhere EBF 12.0.1.3867 for Windows

Current builds for the active platforms...

HP-UX     12.0.1.3798 EBF           29 Oct 2012
 Itanium  11.0.1.2879 EBF           31 Oct 2012

IBM AIX   12.0.1.3798 EBF           24 Oct 2012
          11.0.1.2879 EBF           29 Oct 2012

Linux     16.0.0.1324 GA            05 Mar 2013
          12.0.1.3853 EBF           19 Mar 2013
          11.0.1.2913 EBF           19 Dec 2012

Mac OS    12.0.1.3853 EBF           22 Feb 2013
          11.0.1.2449 EBF           29 Jun 2010

Solaris   12.0.1.3798 EBF           24 Oct 2012
 SPARC    11.0.1.2913 EBF (SA)      19 Dec 2012

Solaris   12.0.1.3798 EBF           29 Oct 2012
 x64      11.0.1.2879 EBF           29 Oct 2012

Windows   16.0.0.1324 GA            05 Mar 2013
          12.0.1.3867 EBF       *** 22 Mar 2013 ***
          12.0.1 French Docs,       25 Sep 2012
                 English Docs,      25 Sep 2012
                 German Docs        25 Sep 2012
          11.0.1.2913 EBF           21 Dec 2012

Other Stuff...

 Older EBFs

 Free support! Q&A forum
   ...or, call Tech Support

 SQL Anywhere...
   ...Sybase home page 
   ...SAP home page 
   ...SAP Developer Center 

 Buy SQL Anywhere 

 Developer Edition... 
   [16.0] [12.0.1] [11.0.1]

 Download the...
   Educational Edition 
   Web Edition 

 Supported Platforms...
   SQL Anywhere 
   Linux 
   OnDemand

 ODBC Drivers for MobiLink

The three asterisks "***" show which Express Bug Fixes (EBFs) and GA builds have appeared on the website since the previous version of this page.
  • Only the latest fully-supported versions of SQL Anywhere (11.0.1, 12.0.1 and 16.0) are shown here.

  • Just because an older version or different platform isn't "fully supported" any more doesn't mean you can't download files (or ask questions, or get help), it just means there won't be any more new EBFs released.


Thursday, March 21, 2013

Latest SQL Anywhere EBF: 12.0.1.3853 for Linux

Current builds for the active platforms...

HP-UX     12.0.1.3798 EBF           29 Oct 2012
 Itanium  11.0.1.2879 EBF           31 Oct 2012

IBM AIX   12.0.1.3798 EBF           24 Oct 2012
          11.0.1.2879 EBF           29 Oct 2012

Linux     16.0.0.1324 GA        *** 05 Mar 2013 ****
          12.0.1.3853 EBF       *** 19 Mar 2013 ****
          11.0.1.2913 EBF           19 Dec 2012

Mac OS    12.0.1.3853 EBF           22 Feb 2013
          11.0.1.2449 EBF           29 Jun 2010

Solaris   12.0.1.3798 EBF           24 Oct 2012
 SPARC    11.0.1.2913 EBF (SA)      19 Dec 2012

Solaris   12.0.1.3798 EBF           29 Oct 2012
 x64      11.0.1.2879 EBF           29 Oct 2012

Windows   16.0.0.1324 GA        *** 05 Mar 2013 ****
          12.0.1.3851 EBF           22 Feb 2013
          12.0.1 French Docs,       25 Sep 2012
                 English Docs,      25 Sep 2012
                 German Docs        25 Sep 2012
          11.0.1.2913 EBF           21 Dec 2012

Other Stuff...

 Older EBFs

 Free support! Q&A forum
   ...or, call Tech Support

 SQL Anywhere...
   ...Sybase home page 
   ...SAP home page 
   ...SAP Developer Center 

 Buy SQL Anywhere 

 Developer Edition... 
   [16.0.0] [12.0.1] [11.0.1]

 Download the...
   Educational Edition 
   Web Edition 

 Supported Platforms...
   SQL Anywhere 
   Linux 
   OnDemand

 ODBC Drivers for MobiLink

The three asterisks "***" show which Express Bug Fixes (EBFs) and GA builds have appeared on the website since the previous version of this page.
  • Only the latest fully-supported versions of SQL Anywhere (11.0.1, 12.0.1 and 16.0.0) are shown here.

  • Just because an older version or different platform isn't "fully supported" any more doesn't mean you can't download files (or ask questions, or get help), it just means there won't be any more new EBFs released.


Monday, March 18, 2013

Reporting and the Audit Trail

Index to all the "Audit Trail" posts
February 23, 2008 Tip: Triggering an Audit Trail
February 28, 2009 Revisited: Triggering an Audit Trail
March 24, 2009 ALTER TABLE and the Audit Trail
March 18, 2013 Reporting and the Audit Trail

One of the most popular articles on this blog was published in February 2008: Tip: Triggering an Audit Trail.

Let's say you want to keep a complete record of every INSERT, UPDATE and DELETE made to a particular table. Furthermore, let's say you want to store this audit trail in the database itself to make it easy to write queries.

You can use SQL Anywhere's CREATE TRIGGER feature to do all that, and with the help of several other features like INSERT WITH AUTO NAME the code becomes quite simple.

[ ...more ]
A year later, after some real-world experience, the code was rewritten and the article republished: Revisited: Triggering an Audit Trail.

And shortly after that, the question "How do I add a column to a table being audited?" was answered in ALTER TABLE and the Audit Trail.

Now it's four years since the original article and the technique is still working, still answering those "What the heck happened?" questions that crop up from time to time.

Not just bug hunting, however; the audit trail can answer questions like "What exactly was in the inventory back on December 31, 2012? The accountants are asking..."
  • "Didn't you remember to take a snapshot of the inventory at midnight on December the 31st?"
    ...nope, forgot

  • "Didn't you build something into the application to record the year-end inventory?"
    ...nope, that didn't happen either

  • "Can't you restore the last backup taken on December 31 and run a query?"
    ...nope, don't keep them around that long
What can be done, and it requires no more effort (or planning) than to write a SELECT, is to use the data in the audit trail to compute the state of the database at year end.

Here's what the tables look like, first the table holding the current inventory, and then the shadow table holding 17,000 before-and-after-images of rows in the item table:
CREATE TABLE item ( -- 873 rows, 3.7M total = 152k table + 3.5M ext + 32k index, 4,401 bytes per row
   sku                      VARCHAR ( 20 ) NOT NULL,
   sku_integer_suffix       UNSIGNED INT NOT NULL 
                               COMPUTE ( integer_suffix(sku) ),
   show_on_website          VARCHAR ( 1 ) NOT NULL DEFAULT 'N' 
                               CONSTRAINT ASA90 CHECK (  
                                  show_on_website in( 'Y','N' )  ),
   usd_price                DECIMAL ( 11, 2 ) NOT NULL DEFAULT 0.0,
   usd_shipping             DECIMAL ( 11, 2 ) NOT NULL DEFAULT 0.0,
   cad_shipping             DECIMAL ( 11, 2 ) NOT NULL DEFAULT 0.0,
   featured                 VARCHAR ( 1 ) NOT NULL DEFAULT 'N' 
                               CONSTRAINT ASA91 CHECK (  
                                  featured in( 'Y','N' )  ),
   stock                    INTEGER NOT NULL DEFAULT 1,
   main_category            VARCHAR ( 50 ) NOT NULL,
   updated_at               TIMESTAMP NOT NULL DEFAULT timestamp,
   title                    VARCHAR ( 100 ) NOT NULL DEFAULT '',
   text_description         LONG VARCHAR NOT NULL DEFAULT '',
   active                   VARCHAR ( 1 ) INLINE 1 PREFIX 1 NOT NULL DEFAULT 'Y' 
                               CONSTRAINT ASA92 CHECK (  
                                  active in( 'Y','N' )  ),
   display_order            BIGINT NOT NULL DEFAULT 0,
   promotional_message      LONG VARCHAR INLINE 256 PREFIX 8 NOT NULL DEFAULT '',
   promotional_message_at   TIMESTAMP NOT NULL DEFAULT '1900-01-01',
   CONSTRAINT ASA93 PRIMARY KEY ( -- 16k
      sku )
 );

CREATE TABLE logged_item ( -- 17,073 rows, 20.5M total = 3.3M table + 17M ext + 160k index, 1,260 bytes per row
   log_id                   UNSIGNED BIGINT NOT NULL DEFAULT autoincrement,
   logged_action            VARCHAR ( 50 ) NOT NULL 
                               CONSTRAINT ASA94 CHECK ( logged_action in( 
                                  'after INSERT', 
                                  'before UPDATE', 
                                  'after UPDATE', 
                                  'before DELETE' )  ),
   logged_at                TIMESTAMP NOT NULL DEFAULT timestamp,
   sku                      VARCHAR ( 20 ) NULL,
   sku_integer_suffix       UNSIGNED INT NULL,
   show_on_website          VARCHAR ( 1 ) NULL,
   usd_price                DECIMAL ( 11, 2 ) NULL,
   usd_shipping             DECIMAL ( 11, 2 ) NULL,
   cad_shipping             DECIMAL ( 11, 2 ) NULL,
   featured                 VARCHAR ( 1 ) NULL,
   stock                    INTEGER NULL,
   main_category            VARCHAR ( 50 ) NULL,
   updated_at               TIMESTAMP NULL,
   title                    VARCHAR ( 100 ) NULL,
   text_description         LONG VARCHAR NULL,
   active                   VARCHAR ( 1 ) NULL,
   display_order            BIGINT NULL,
   promotional_message      LONG VARCHAR NULL,
   promotional_message_at   TIMESTAMP NULL,
   CONSTRAINT ASA95 PRIMARY KEY ( -- 160k
      log_id )
 );
Because of the way the audit trail triggers work, the rows in logged_item with the same sku (stock keeping unit) value can be ordered on log_id to give the complete chronological history of that sku...

...and the last row inserted before January 1, 2013 tells you the state of that sku in the year-end inventory.

A perfect application for the LAST_VALUE() function and the WINDOW clause, right?
SELECT LAST_VALUE ( log_id ) OVER sku_window AS last_log_id
  FROM logged_item
 WHERE logged_at < '2013-01-01'
WINDOW sku_window AS ( PARTITION BY sku 
                       ORDER BY log_id )
 ORDER BY last_log_id;
  • The WHERE clause grabs everything in logged_item up to midnight on December 31, 2012,

  • the PARTITION BY sku clause creates a separate partition in the window for each sku,

  • the inner ORDER BY log_id clause sorts the partition so the LAST_VALUE ( log_id ) function will return the last row in the chronological history of each partition,

  • the "LAST_VALUE ( log_id ) OVER sku_window AS last_log_id" select-list entry computes the last log_id for each sku and gives it an alias name "last_log_id", and

  • the outer ORDER BY last_log_id clause sorts the final result set.
Alas, the results are worse-than-useless, just the numbers 1, 2, 3:
last_log_id
----------- 
1
2
3
4
5
6
7
8
9
10
11
12
...
16804
16805
16806
Heck, I can do that by calling sa_rowgenerator!

It turns out that LAST_VALUE() needs the WINDOW clause to have a full-tilt-boogie RANGE BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING specification (the docs say that's the default but reality intrudes):

SELECT LAST_VALUE ( log_id ) OVER sku_window AS last_log_id
  FROM logged_item
 WHERE logged_at < '2013-01-01'
WINDOW sku_window AS ( PARTITION BY sku 
                       ORDER BY log_id
                       RANGE BETWEEN UNBOUNDED PRECEDING 
                                 AND UNBOUNDED FOLLOWING )
 ORDER BY last_log_id;
The result isn't much better, still WAY too many rows (16,806), but at least they look different:
last_log_id          
----------- 
297                  
297                  
297                  
369                  
369                  
369                  
1084                 
1084                 
1084                 
1164                 
1164                 
1164       
...          
16806
16806
16806
At this point, one must have faith: the WINDOW clause works, and so does LAST_VALUE(), and they aren't just powerful, they are fast too!

The problem here is it's returning one row for every row in logged_item, and those rows repeat the LAST_VALUE() for each partition. That's the way partitions work, it's something you (I) must get used to.

...unlike the RANGE nonsense, which may forever remain a mystery. Like waving a dead chicken over the keyboard, if your WINDOW query doesn't work try RANGE BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING!
What's the obvious solution to duplicate rows?

Why, the DISTINCT keyword, of course!

...one of the most loved, most used, most mis-used dead chickens in the SQL toolbox!
Here it is, solving the repeating-LAST_VALUE problem:
SELECT DISTINCT LAST_VALUE ( log_id ) OVER sku_window AS last_log_id
  FROM logged_item
 WHERE logged_at < '2013-01-01'
WINDOW sku_window AS ( PARTITION BY sku 
                       ORDER BY log_id
                       RANGE BETWEEN UNBOUNDED PRECEDING 
                                 AND UNBOUNDED FOLLOWING )
 ORDER BY last_log_id;
Now instead of 16,806 rows there are only 856, one row per sku:
last_log_id          
----------- 
297                  
369                  
1084                 
1164                 
...
16806
Each row in that result set points to a single row in logged_item, and that row shows the last action before year end for the corresponding item row:
  • logged_action = 'after INSERT' for an item that was inserted before year-end,

  • logged_action = 'after UPDATE' for an item that was updated before year-end, and

  • logged_action = 'before DELETE' for an item that was deleted before year-end.
There are no entries where logged_action = 'before UPDATE' because that action is always followed by an 'after UPDATE' with the same logged_at timestamp (once again, that's the way the audit trail triggers work).

In other words, that query embodies the year-end snapshot that accounting wants, and it can be made useful as a view:

CREATE VIEW year_end_2012_logged_item AS 
   ( SELECT DISTINCT LAST_VALUE ( log_id ) OVER sku_window AS last_log_id
       FROM logged_item
      WHERE logged_at < '2013-01-01'
     WINDOW sku_window AS ( PARTITION BY sku 
                            ORDER BY log_id
                            RANGE BETWEEN UNBOUNDED PRECEDING 
                                      AND UNBOUNDED FOLLOWING ) );
Here's an example of how that view was used to summarize the year-end inventory by category:
SELECT logged_item.main_category                           AS Category,
       COUNT(*)                                            AS SKU_count,
       SUM ( logged_item.stock )                           AS item_count,
       SUM ( logged_item.stock * logged_item.usd_price )   AS inventory_value
  FROM logged_item
          INNER JOIN year_end_2012_logged_item
                  ON year_end_2012_logged_item.last_log_id = logged_item.log_id
 WHERE logged_item.logged_action   IN ( 'after INSERT', 'after UPDATE' )
   AND logged_item.show_on_website = 'Y'
   AND logged_item.main_category   <> 'BOOKS'
 GROUP BY logged_item.main_category
 ORDER BY logged_item.main_category;

Category             SKU_count   item_count  inventory_value                 
-------------------- ----------- ----------- ---------------
COLLECTIBLES         18          18          4409.00
GIFTWARE             19          25          663.00
LINENS & TEXTILES    48          48          1804.00
POTTERY & GLASS      61          152         5566.00
SILVERWARE           106         111         4342.00
  • The INNER JOIN uses the year_end_2012_logged_item view to identify which logged_item rows form the year-end snapshot,

  • the WHERE eliminates the deletions, and applies two business-related predicates (include active items, exclude books), and

  • the GROUP BY enables the COUNT() and SUM() calculations.
It's worth noting that the item table doesn't take part in any of these queries, nor can it because it represents the current state of inventory and accounting only cares about year end.

Other queries can be written, to show detail or summaries, by using this template:

SELECT [whatever columns and/or aggregate function calls you want]
  FROM logged_item
          INNER JOIN year_end_2012_logged_item
                  ON year_end_2012_logged_item.last_log_id = logged_item.log_id
 WHERE logged_item.logged_action   IN ( 'after INSERT', 'after UPDATE' )
   AND [whatever other predicates you want]
 [plus whatever GROUP BY and ORDER BY clauses you might need]
Suddenly, the audit trail tables and triggers are justified not only because of safety and security, but because they provide significant reporting support with zero extra effort.

Monday, March 11, 2013

SQL Anywhere 16 Synchronization Webcast

Tom Slee's webcast Discover the New Data Synchronization Features in SAP Sybase SQL Anywhere 16 is two days from today, on Wednesday, March 13, 2013 at 1:00 PM EST...


During this Webcast, you’ll see live demos and hear Tom Slee, product manager at SAP Canada, discuss new data synchronization highlights and benefits.

You’ll also learn about:
  • Improved synchronization and mobile database performance features that help ensure best-in-class performance of your mobile apps – regardless of deployment platform

  • Innovative profiling tools that allow you to drill down into your synchronization environment – so you can identify potential issues quickly and efficiently

  • Enhanced design and development tools that enable you to more easily build and deploy custom-designed synchronization scripts

  • Greater support for data synchronization with SAP HANA – allowing you to easily extend your next-generation in-memory applications to mobile environments
This is your opportunity to learn about all that SAP Sybase SQL Anywhere 16.0 has to offer in data synchronization.

We hope you’ll attend this valuable Webcast.


Some other links...



Friday, March 8, 2013

Tuesday, March 5, 2013

It's Here! SQL Anywhere 16 Developer Edition Download

Ready for download now: SAP Sybase SQL Anywhere 16 Developer Edition Registration.

Also available: The docs for SQL Anywhere 16.

Coming soon: Jason Hinsperger's webcast Data Management Features in SAP Sybase SQL Anywhere 16.0 on Wednesday, March 6, 2013 1:00 PM EST.


SQL Anywhere 16 Webcast and Docs

The docs for SQL Anywhere 16 are now online, can the actual software be far behind?

In the meantime, you can attend Jason Hinsperger's webcast Data Management Features in SAP Sybase SQL Anywhere 16.0 on Wednesday, March 6, 2013 1:00 PM EST.



Monday, March 4, 2013

SQL Anywhere 16 Sneak Peek: Abort, Retry, Escalate

Today marks the second anniversary of a promise reported in this request for "Assertion Relief":

"...we plan to make changes to allow corrupted databases to be stopped without bringing down the server. Printing database information to related assertion failures should be part of that."

Ta Daaaa!


Relief is here, now, with SQL Anywhere 16's new dbsrv16 -ufd abort, restart, escalate option that specifies "the action that the database server takes when a fatal error or assertion failure occurs on a database."

Well, it's ALMOST here, and will be as soon as SAP sets SQL Anywhere 16 loose.

The dbsrv16 -ufd option applies to database fatal errors and assertions, not server errors... if you get an error related to the server itself, then presumably the server still stops, just like it does now no matter which kind of fatal error you get.

Caveat Emptor: The word "presumably" is shorthand for "I have not seen dbsrv16 -ufd in operation yet"... eventually, I will... I get more than enough assertion errors without having to make one happen on purpose.

But, for now, this article is really just me reading the Help to you :)

Here are the choices:
  • dbsrv16 -ufd abort "The affected database is shut down. The statuses of the database server and other databases remain unchanged."

    This is the new default, which is different from the current behavior in Version 12 (see "escalate" below). This setting makes sense if you're really not expecting assertions, and you want to kick the users offline until you fix the database.

  • dbsrv16 -ufd restart "The affected database is shut down with an attempt to restart the database. If the restart attempt fails, a database server assertion failure is raised."

    This setting makes sense for lights-out operations at both ends of the spectrum: A single assertion in one database among hundreds on a SQL Anywhere server doesn't cause them all to halt, and an embedded database application may be designed so the user can seamlessly reconnect and carry on past a transient database assertion. The latter is what Foxhound's going to do in its next release... and, I think, it's going to be a popular choice for a lot of people.

  • dbsrv16 -ufd escalate "The database assertion failure or fatal error is treated as a database server assertion failure or fatal error."

    That is the Old Way Of Doing Things... when a light burns out in the lavatory on the flight from LA to Seoul, the whole fleet of aircraft shuts down and crashes.
The fact that concerns about backward compatiblity didn't result in escalate being the new default is further proof of the saying "Watcom does things the way they should be done!"



Friday, March 1, 2013

SQL Anywhere 12.0.1 Sneak Peek: Compare Databases

Yes, yes, SQL Anywhere 12.0.1 is two years old so this is hardly a "Sneak Peak".

But, this feature is new to me so maybe you missed it too:


Support added for comparing database schemas and making them the same

You can use Sybase Central to compare two databases. The comparison generates SQL statements that you can review to determine the differences between two databases. You can execute the SQL statements to make the one database the same as the other database.



I like the "making them the same" part... let's see how it all works:



The "Objects" tab lists all the bits and pieces in the database, from tables down to user ids and unique constraints:
  • The filter-as-you-type "Search" field is very fast,

  • which more than makes up for the strange default sort order,

  • plus you can change the sort order by clicking on the column headings, and

  • clicking on a matching pair in the top frame brings up a side-by-side comparison of those objects in the bottom frame.
The "SQL Script" tab presents a dbunload-style script that changes all the objects in Database 1 to look (more or less) like the objects in Database 2:



Without a Search field, however, the SQL Script display is singularly useless... it's way too big to scroll through, and even if you find what you're looking for you can't do anything with it (no select, so no copy and paste).

What you CAN do, however, is press the Save As... button and then use your favorite editor (ISQL, Wordpad, whatever) to yank out the bits you want, like this:
ALTER TABLE "DBA"."rroad_group_1_property_pivot" ADD "CPU_count" integer NOT NULL COMPUTE (case when COALESCE(NumLogicalProcessorsUsed,0) > 0 then NumLogicalProcessorsUsed
when COALESCE(NumProcessorsAvail,0) > 0 then NumProcessorsAvail
else 1
end)
go

ALTER TABLE "DBA"."rroad_group_1_property_pivot" ADD "autodropped_connection_count" integer NOT NULL DEFAULT 0
go

CREATE INDEX "ix_number_DESC_id_lost" ON "DBA"."rroad_group_1_property_pivot"
    ( "sample_set_number" DESC,"sampling_id","sample_lost" )
go

CREATE INDEX "ix_id_lost" ON "DBA"."rroad_group_1_property_pivot"
    ( "sampling_id","sample_lost" )
go
There are some restrictions, like it only works on database files created with SQL Anywhere 10 or later, plus those databases have to be running on SQL Anywhere 12 engines, and it won't change the order of columns in a table:
ALTER TABLE "DBA"."rroad_odbc_dsn" DROP PRIMARY KEY
go

ALTER TABLE "DBA"."rroad_odbc_dsn" ADD PRIMARY KEY ("odbc_dsn" ASC,"session_id" ASC)
go

// Can't reorder columns for table "DBA"."rroad_odbc_dsn"
// ("ALTER TABLE ... ADD column-name ... " adds columns to end of table only)
// Database 'f - DBA' (database 1):
// CREATE TABLE "DBA"."rroad_odbc_dsn" (
//     "session_id"                     varchar(36) NOT NULL
//    ,"odbc_dsn"                       varchar(255) NOT NULL
//    ,PRIMARY KEY ("session_id" ASC,"odbc_dsn" ASC) 
// )
// Database 'f - DBA' (database 2):
// CREATE TABLE "DBA"."rroad_odbc_dsn" (
//     "odbc_dsn"                       varchar(255) NOT NULL
//    ,"session_id"                     varchar(36) NOT NULL
//    ,PRIMARY KEY ("odbc_dsn" ASC,"session_id" ASC) 
// )
Other limitations are described here, including some that might involve loss of data.


In SQL Anywhere 16, Compare Databases is pretty much the same except...
  • it's now called "Compare Database Schemas..." to reduce expectations about the data,

  • the password is no longer blanked out (grrr!) every time you open the Connect window, and

  • it now insists you start both databases using SQL Anywhere 16 rather than version 12.
As features go, it's a good start. At the very least, it will be immediately useful for generating those funky ALTER statements needed to deploy changes that were initially coded by DROP and CREATE.

And for double-checking "what's different" between old and new versions of a database... especially when you need to see all the nooks and crannies in the schema.

For daily use, however, not so much... not until it gets some more ease-of-use enhancements, like the ability to search and select in the SQL Script pane, and to change the order of the scripts.

Yes, the generated order of all those ALTER statements is important if you're going to run the whole thing at once, but...

...that's an unlikely use for it.




Wednesday, February 27, 2013

Latest SQL Anywhere EBFs: 12.0.1 for Mac and Windows

Current builds for the active platforms...

HP-UX     12.0.1.3798 EBF           29 Oct 2012
 Itanium  11.0.1.2879 EBF           31 Oct 2012

IBM AIX   12.0.1.3798 EBF           24 Oct 2012
          11.0.1.2879 EBF           29 Oct 2012

Linux     16.0.0.1018 Beta          09 Nov 2012
          12.0.1.3827 EBF (SA)      10 Dec 2012
          12.0.1.3798 EBF           07 Nov 2012
          11.0.1.2913 EBF (SA)      19 Dec 2012
          11.0.1.2879 EBF (All)     03 Jan 2013

Mac OS    12.0.1.3853 EBF       *** 22 Feb 2013 ***
          11.0.1.2449 EBF           29 Jun 2010

Solaris   12.0.1.3798 EBF           24 Oct 2012
 SPARC    11.0.1.2913 EBF (SA)      19 Dec 2012

Solaris   12.0.1.3798 EBF           29 Oct 2012
 x64      11.0.1.2879 EBF           29 Oct 2012

Windows   16.0.0.1018 Beta          09 Nov 2012
          12.0.1.3851 EBF       *** 22 Feb 2013 ***
          12.0.1 French Docs,       25 Sep 2012
                 English Docs,      25 Sep 2012
                 German Docs        25 Sep 2012
          11.0.1.2913 EBF           21 Dec 2012

Other Stuff...

 SQL Anywhere 16 Beta

 Older EBFs

 Free support! Q&A forum
   ...or, call Tech Support

 SQL Anywhere...
   ...Sybase home page 
   ...SAP home page 
   ...SAP Developer Center 

 Buy SQL Anywhere 

 Developer Edition... 
   [12.0.1] [11.0.1]

 Download the...
   Educational Edition 
   Web Edition 

 Supported Platforms...
   SQL Anywhere 
   Linux 
   OnDemand

 ODBC Drivers for MobiLink

The three asterisks "***" show which Express Bug Fixes (EBFs) and Beta builds have appeared on the website since the previous version of this page.
  • Only EBFs for the latest fully-supported versions of SQL Anywhere (11.0.1 and 12.0.1) are shown here, plus Beta builds for 16.0.0.

  • Just because an older version or different platform isn't "fully supported" any more doesn't mean you can't download files (or ask questions, or get help), it just means there won't be any more new EBFs released.


Sunday, February 24, 2013

Taming A Runaway Temporary File

A temporary file, also known as a temporary dbspace, is automatically created whenever a SQL Anywhere database is started, and it is automatically deleted whenever that database is stopped. The files are named sqla0000.tmp, sqla0001.tmp and so on, and by default (on Windows 7) they are located in the C:\Users\[user-name]\AppData\Local\Temp folder:

 Directory of C:\Users\Breck\AppData\Local\Temp

02/23/2013  10:04 AM        26,353,664 sqla0000.tmp
02/23/2013  10:52 AM           200,704 sqla0001.tmp
02/23/2013  10:55 AM           102,400 sqla0003.tmp
The temporary file is used for temporary data. Exactly what constitutes "temporary data" is more or less a mystery, as are the conditions under which temporary data is written to the temporary file rather than stored in the database cache. You have control over the temporary file location, but the SQL Anywhere server has complete control over how, why and when the temporary file is used.

In terms of data nothing of lasting value is ever stored in these files. If for some reason (say, after a crash) one of these files remains in existence, it's just wasting space and you can go ahead and delete it. In case of doubt, delete: if it's still in use by SQL Anywhere you won't be able to.

In terms of performance, the temporary file is usually no problem. In fact, at most SQL Anywhere shops the temporary file is completely invisible: no one even knows it's there.

It is possible, however, for runaway temporary file usage to cause grief:
  • If SQL Anywhere decides to write the temporary data to the file instead of the database cache, excessive disk I/O may be the result.

  • If the temporary file is located on the same physical drive as other database files, the disk I/O may not be efficient.

  • Prior to SQL Anywhere 10, a runaway process could cause the server to crash because the drive holding the temporary file ran out of space. With version 10 and later, a runaway process is prevented from using more than 80% of the remaining disk space, which on today's enormous disk drives means...

  • ...if not a crash, then at least "Holy Cow!" moment when first discovering that sqla0000.tmp has grown to 500G for a database that is only 300M in size.
It's easy to tell SQL Anywhere where to put the temporary file: just use the dbsrv12 -dt option or one of the SATMP, TMP, TMPDIR or TEMP environment variables.

The next step, telling SQL Anywhere to stop runaway connections from using too much temporary space, is harder. Here's how...

Step 1: Put a limit on temporary space usage


Pick a limit, say 512M, on the amount of temporary storage used by each individual connection, and put that limit into effect:
SET OPTION PUBLIC.temp_space_limit_check = 'On';

SET OPTION PUBLIC.max_temp_space = '512M';
Double-check that these settings apply to end-user connections. One method is to have the end user connect via dbisql, run the SET command (no parameters, just "SET") and report the values displayed:





Step 2: Test the limit check


Here's command to start a connection named "adhoc-queries" that will be used to test the limit check:
"%SQLANY12%\bin32\dbisql.com"^
  -c "ENG=inventory_envy;DBN=inventory;UID=k.delacruz;PWD=sql;CON=adhoc-queries"
Here's a query that displays the size of the temporary file, plus the temporary space currently used by "adhoc-queries"; this query should be run on some OTHER connection:
SELECT STRING (
          CAST ( CAST ( DB_EXTENDED_PROPERTY ( 'FileSize', 'temporary' ) AS INTEGER )
             * 4096.0 / ( 1024 * 1024 )
             AS DECIMAL ( 11, 2 ) ),
          'M' ) 
          AS "Temporary FileSize",
       STRING ( 
          CAST ( CAST ( CONNECTION_PROPERTY ( 'TempfilePages', Number ) AS INTEGER )
             * 4096.0 / ( 1024 * 1024 )
              AS DECIMAL ( 11, 2 ) ),
          'M' ) 
          AS "TempFilePages"
  FROM sa_conn_properties() 
 WHERE PropName = 'Name' 
   AND Value = 'adhoc-queries'

Temporary FileSize,TempFilePages
'1.04M','.11M'
So far so good... but here's a query to run on the "adhoc-queries" connection that will cause the temporary space usage to soar:
SELECT a.*
  INTO #temp_inventory
  FROM inventory AS a,
       inventory AS b;
It's self-join between a million-row table and itself, and because the WHERE clause has been omitted (a common error with adhoc queries) it is effectively a CROSS JOIN.

In testing that query took less than a minute to reach the max_temp_space limit:
Temporary FileSize,TempFilePages
'41.04M','35.90M'

Temporary FileSize,TempFilePages
'462.97M','461.87M'
When the max_temp_space limit kicked in, SQL Anywhere stopped the SELECT INTO and started rolling it back. The connection-level TempFilePages value started to sink, but the temporary file size remained at the "high water mark":
Temporary FileSize,TempFilePages
'526.97M','302.60M'
Eventually, the failing connection received a SQLCODE -1000 error message, but not until the connection-level TempFilePages had sunk back to pre-SELECT INTO levels:
Could not execute statement.
Temporary space limit exceeded
SQLCODE=-1000, ODBC 3 State="HY000"

Temporary FileSize,TempFilePages
'526.97M','.23M'
Here's what the total temporary space usage looked like from Foxhound's point of view:


  • At first (bottom line), before the runaway SELECT INTO started executing, the total "Temp Space" amount was 760K.

  • In less than a minute (6th line up from the bottom), the amount reached 503M.
    At this point, the "adhoc-queries" connection (see the lower frame in the image below) was responsible for 502M of the total:

  • The next sample (7th line up from the bottom of the first image) shows the total amount has started to drop (463M). At this point SQL Anywhere has cancelled the SELECT INTO operation and started to roll it back.

  • Two minutes later (2nd line from the top), SQL Anywhere has finished cancelling the SELECT INTO.
    The total Temp Space is back down to 868K, and at 3:50:18 the SQLCODE -1000 error was returned to the "adhoc-queries" connection:



"Why bother setting a limit, just increase the database cache."


Here's why: In the test above, the dbsrv12 -c 1G -ch 2G options were used even though the entire database file was only 288M. The database was completely idle except for the runaway SELECT INTO test, yet SQL Anywhere chose to grow the temporary file to 527M instead of using (or growing) the cache.

Hence the earlier comment about the conditions under which temporary data is written to the temporary file being "more or less a mystery".


Wednesday, February 20, 2013

SQL Anywhere 16 Sneak Peek: ISQL Text Completion

The ISQL Text Completion feature in SQL Anywhere 16 has been improved in three ways:

  • it shows the parameter list when you type the opening "(" of a procedure or function call,

  • it shows the closing bracket when you type an opening bracket, and

  • it shows the closing quote when you type an opening quote.
Here's what the first two look look like:



The parameter list completion works for user-defined functions as well:
CREATE FUNCTION f (
   in parm1 INTEGER,
   in parm2 VARCHAR ( 100 ) )
   RETURNS VARCHAR ( 100 )
BEGIN
   RETURN 'Hello';
END;


The bracket completion works for all three "{", "[" and "(", and the quote completion works for double and single quotes:



To set your preferences for text completion see the ISQL Tools - Options... - Editor - Text Completion dialog box:




Friday, February 15, 2013

Product Suggestion: DEPENDENT AUTOINCREMENT

You're probably familiar with DEFAULT AUTOINCREMENT which can be used to very simply, efficiently and safely initialize a numeric primary key column with the sequence 1, 2, 3, ...

If you use SQL Remote or MobiLink synchronization you're probably also familiar with DEFAULT GLOBAL AUTOINCREMENT which creates the partitioned sequence

  • 1, 2, 3, ... for a database with SET OPTION PUBLIC.global_database_id = '0',

  • 10000001, 10000002, 10000003, ... for a database with global_database_id = '1',

  • 20000001, 20000002, 20000003, ... for a database with global_database_id = '2', and so on,
so that a primary key column can be globally unique across hundreds or thousands of separate databases.

But what about initializing columns in dependent tables, like line_number 1, 2, 3 within order_number 1, then line_number 1, 2, 3 again within order_number 2?

Suggestion: DEFAULT DEPENDENT AUTOINCREMENT

The DEFAULT DEPENDENT AUTOINCREMENT ( column-name ) clause would initialize the column to values 1, 2, 3 within each distinct value of another column-name in the same table, like this:

CREATE TABLE parent (
   pkey   INTEGER NOT NULL DEFAULT AUTOINCREMENT PRIMARY KEY,
   data   INTEGER NOT NULL );

CREATE TABLE child (
   fkey   INTEGER NOT NULL REFERENCES parent ( pkey ),
   dkey   INTEGER NOT NULL DEFAULT DEPENDENT AUTOINCREMENT ( fkey ),
   data   INTEGER NOT NULL,
   PRIMARY KEY ( fkey, dkey ) );

BEGIN
   DECLARE @pkey INTEGER;
   INSERT parent VALUES ( DEFAULT, 1 );
   SET @pkey = @@IDENTITY;
   INSERT child VALUES ( @pkey, DEFAULT, 10 );
   INSERT child VALUES ( @pkey, DEFAULT, 20 );
   INSERT parent VALUES ( DEFAULT, 2 );
   SET @pkey = @@IDENTITY;
   INSERT child VALUES ( @pkey, DEFAULT, 30 );
   INSERT child VALUES ( @pkey, DEFAULT, 40 );
   COMMIT;
   SELECT * FROM parent ORDER BY pkey;
   SELECT * FROM child ORDER BY fkey, dkey;
END;

pkey        data        
----------- ----------- 
1           1           
2           2           


fkey        dkey        data        
----------- ----------- ----------- 
1           1           10          
1           2           20          
2           1           30          
2           2           40          

As with other kinds of AUTOINCREMENT columns, the @@IDENTITY connection-level variable would return the most recent value calculated across all columns; i.e, in the example above, @@IDENTITY would contain the successive values 1, 1, 2, 2, 1, 2 after each of the six INSERT statements.


Monday, February 11, 2013

SQL Anywhere 16 Sneak Peek: xp_getenv()

Now you can get the value of server-side environment variables like PATH and LOCALAPPDATA and TEMP inside your SQL scripts, even inside stored procedures, triggers and web services.

When you call xp_getenv ( 'variable' ) from a SQL Anywhere 16 database running on Windows 7, it works for

  • all the environment variables that appear when you run the SET command at the command prompt on the database server,

  • which is a superset of the variables in the Control Panel - All Control Panel Items - System - Advanced System Settings - Environment Variables window,

  • which also includes custom SET WHATEVER values you have defined before starting the database engine,

  • but xp_getenv() doesn't work for the dynamic environment variables like CD and ERRORLEVEL; e.g., %CD% returns the current directory inside a batch file but xp_getenv ( 'CD' ) returns NULL,

  • nor does xp_getenv() work for custom SET variables that fall out of scope before xp_getenv() is called; e.g., the following SELECT returns NULL:
       CALL xp_cmdshell ( 'SET WHATEVER=123' );
       SELECT xp_getenv ( 'WHATEVER' );
    
In other words, environment variables are just a teeny bit mysterious, so check your assumptions at the door and test your xp_getenv calls.

But... but... xp_getenv() is still cool...


Here's how it works:
SELECT xp_getenv ( 'APPDATA' );

xp_getenv('APPDATA')
--------------------------------------------------------------
0x433a5c55736572735c427265636b5c417070446174615c526f616d696e67
Oops, xp_getenv() returns a LONG BINARY string, so CAST is your friend when you're using ISQL:
SELECT CAST ( xp_getenv ( 'APPDATA' ) AS VARCHAR );

xp_getenv('APPDATA')
------------------------------
C:\Users\Breck\AppData\Roaming        
If you need to search the PATH list, you can combine sa_split_list() with xp_getenv() to break it down into its component parts like this:
SELECT row_value AS "Path"
  FROM sa_split_list ( CAST ( xp_getenv ( 'PATH' ) AS VARCHAR ), ';' )
 ORDER BY line_num;

Path
-----------------------------------------------------------------
C:\Program Files\Common Files\Microsoft Shared\Windows Live
C:\Program Files (x86)\Common Files\Microsoft Shared\Windows Live
C:\Windows\system32
C:\Windows
...
C:\Program Files (x86)\Sybase\Shared\win32
C:\Program Files (x86)\Sybase\Shared\Sybase Central 4.3\win32
C:\Program Files\SQL Anywhere 16\bin64
C:\Program Files\SQL Anywhere 16\bin32
Here's an example of a server startup script that uses a custom SET VCD=%CD% command to create a copy of the CD dynamic environment variable that (unlike CD) is available to xp_getenv():
SET VCD=%CD%

"%SQLANY16%\bin64\dbspawn.exe"^
  -f "%SQLANY16%\bin64\dbsrv16.exe"^
  ddd16.db 

SELECT CAST ( xp_getenv ( 'VCD' ) AS VARCHAR );

xp_getenv('VCD')
----------------
C:\data\xpdemo


Thursday, February 7, 2013

Latest SQL Anywhere EBFs for January 2013

Current builds for the active platforms...

HP-UX     12.0.1.3798 EBF           29 Oct 2012
 Itanium  11.0.1.2879 EBF           31 Oct 2012

IBM AIX   12.0.1.3798 EBF           24 Oct 2012
          11.0.1.2879 EBF           29 Oct 2012

Linux     16.0.0.1018 Beta          09 Nov 2012
          12.0.1.3827 EBF (SA)      10 Dec 2012
          12.0.1.3798 EBF           07 Nov 2012
          11.0.1.2913 EBF (SA)      19 Dec 2012
          11.0.1.2879 EBF (All) *** 03 Jan 2013 ***

Mac OS    12.0.1.3819 EBF           10 Dec 2012
          11.0.1.2449 EBF           29 Jun 2010

Solaris   12.0.1.3798 EBF           24 Oct 2012
 SPARC    11.0.1.2913 EBF (SA)      19 Dec 2012

Solaris   12.0.1.3798 EBF           29 Oct 2012
 x64      11.0.1.2879 EBF           29 Oct 2012

Windows   16.0.0.1018 Beta          09 Nov 2012
          12.0.1.3840 EBF       *** 28 Jan 2013 ***
          12.0.1 French Docs,       25 Sep 2012
                 English Docs,      25 Sep 2012
                 German Docs        25 Sep 2012
          11.0.1.2913 EBF           21 Dec 2012

Other Stuff...

 SQL Anywhere 16 Beta

 Older EBFs

 Free support! Q&A forum
   ...or, call Tech Support

 SQL Anywhere home page 

 Buy SQL Anywhere 

 Developer Edition... 
   [12.0.1] [11.0.1]

 Download the...
   Educational Edition 
   Web Edition 

 Supported Platforms...
   SQL Anywhere 
   Linux 
   OnDemand

 ODBC Drivers for MobiLink

The three asterisks "***" show which Express Bug Fixes (EBFs) and Beta builds have appeared on the website since the previous version of this page.
  • Only EBFs for the latest fully-supported versions of SQL Anywhere (11.0.1 and 12.0.1) are shown here, plus Beta builds for 16.0.0.

  • Just because an older version or different platform isn't "fully supported" any more doesn't mean you can't download files (or ask questions, or get help), it just means there won't be any more new EBFs released.


Monday, January 7, 2013

Beware Of The OUT Parameter

Question: Why doesn't SQL Anywhere raise a "Wrong number of parameters" exception for the following CALL?

The procedure originally had three parameters @px, @py and @pz. A fourth parameter @pb was added, but the CALL wasn't changed.

According to the docs this should have resulted in an error: "Procedure arguments can be assigned default values in the CREATE PROCEDURE statement, and missing parameters are assigned the default value. If no default is set, and an argument is not provided, an error is given."

...but, in this case, there's no error, just a wrong answer:

CREATE PROCEDURE p (
   IN    @px      INTEGER,
   IN    @pb      INTEGER,
   OUT   @py      INTEGER,
   OUT   @pz      INTEGER )
BEGIN
   SET @py = @px + 1;
   SET @pz = @pb + 99;
END;

BEGIN
DECLARE @x      INTEGER;
DECLARE @y      INTEGER;
DECLARE @z      INTEGER;

SET @x = 1;

CALL p ( @x, @y, @z );

SELECT @x, @y, @z;

END;

@x,@y,@z
1,(NULL),2
Short Answer: Don't use OUT, use INOUT instead, if you want SQL Anywhere to detect missing arguments in your CALL statements.

Long Answer: SQL Anywhere allows varying-length argument lists in CALL statements, and it assumes the omitted argument(s) correspond to parameters at the end of the list.

In other words, SQL Anywhere doesn't know @pb is the missing parameter, it thinks @pz is missing. And, apparently, the statement in the docs "If no default is set, and an argument is not provided, an error is given." doesn't apply to missing OUT arguments.

It should, of course

It makes no sense for an OUT argument to be optional since by definition the procedure is going to assign a value to the corresponding parameter, and there's no place for that value to go if the argument is missing.

Nonetheless, no error is displayed when "SET @pz = @pb + 99;" assigns a value to a parameter with no corresponding argument.

Workaround 1: Use INOUT instead of OUT

As suggested earlier, INOUT is a perfectly good substitute for OUT, no further changes to your code are required: You don't actually have to assign a value before calling the procedure since presumably the code inside the procedure is still going to treat it like an "out" parameter.

Now, if you add a parameter @p to the CREATE PROCEDURE and forget to change the CALL, you'll be told about it right away:
CREATE PROCEDURE p (
   IN    @px      INTEGER,
   IN    @pb      INTEGER,
   INOUT @py      INTEGER,
   INOUT @pz      INTEGER )
BEGIN
   SET @py = @px + 1;
   SET @pz = @pb + 99;
END;

BEGIN
DECLARE @x      INTEGER;
DECLARE @y      INTEGER;
DECLARE @z      INTEGER;

SET @x = 1;

CALL p ( @x, @y, @z );

SELECT @x, @y, @z;

END;

Could not execute statement.
Wrong number of parameters to function 'p'
SQLCODE=-154, ODBC 3 State="42000"

Workaround 2: Put OUT before IN

Another workaround is to code your procedures with all the IN and INOUT parameters after the OUTs. That way, if an argument is missing from the call, SQL Anywhere will think it's a missing IN or INOUT parameter and raise an error:
CREATE PROCEDURE p (
   OUT @py      INTEGER,
   OUT @pz      INTEGER,
   IN  @px      INTEGER,
   IN  @pb      INTEGER )
BEGIN
   SET @py = @px + 1;
   SET @pz = @pb + 99;
END;

BEGIN
DECLARE @x      INTEGER;
DECLARE @y      INTEGER;
DECLARE @z      INTEGER;

SET @x = 1;

CALL p ( @y, @z, @x );

SELECT @x, @y, @z;

END;

Could not execute statement.
Wrong number of parameters to function 'p'
SQLCODE=-154, ODBC 3 State="42000"
Personally, I like Workaround 1 better (use INOUT instead of OUT) because I like coding CALL statements with all the inputs first followed by the outputs.

Better yet, I'd like a product enhancement to require arguments be provided for all OUT parameters.