Search This Blog

Friday, April 16, 2010

Precision: What is it Precisely?

Alex has an interesting post over on Nerds Central which I believe you may find of value.

He details a couple of useful items specific to decimal precision in Cobol and Java.  It is interesting  because it illustrates the use of a Cobol data type I've not played with, float-long. 

Float-Long came about a few years ago, along with Float-Short and Float-Extended and was made part of the 2002 Standard I believe.  To make a long story short, Float-Long isn't quite as accurate as you would hope, but does have its uses.  Alex provides a complete example showing how to acheive both an exact arithmatic answer using a Comp data type and how to use the Float-Long data type.

The other reason I find the post of interest is that in his article, it appears the Java code can only duplicate what the Cobol Float-Long data type provides (which is inaccurate), not the more precise answer he achieved using the more standard Cobol data type of Comp-1. 

Am I mistaken in my interpretation of the Java sample he provided?

This looks like a red flag to me if your Java code is doing any math which requires a high level of accuracy on the right side of the decimal.

Can someone verify this and post a Java-based example confirming/denying the problem?

I'm curious but only know enough Java to be able to nod in the right places when talking with a Java developer. *smile*

P.S.:  The Visual Studio 2010 Launch in Las Vegas was quite the event.  Thanks to various issues with my plane ride (rerouted due to medical emergency and then detained in Albuquerque, NM due to mechanical issues, etc), it only took me 20 hours or so to make the 3 and 1/2 hour trip home.  At least I wasn't flying to London!

2 comments:

  1. Hi,

    For clarity:

    comp-1 is an oddity, the precision is dependant on the COBOL version. This is also the case for comp-2.

    The true high precision stuff comes with comp and comp-5, comp-3. All of these work with base 10 arithmetic not base-2 as float-long and comp-2 (MF) do.

    Java its self as no equivalent to:

    pic s9(18)v(18) comp.

    or

    pic s9(18)v(18) comp-3.

    etc.

    However, there are libraries which allow manipulation of decimal fixed point mathematics. The particular library that is normally used in BigDeciml.

    Because this is not part of the language, using it is much more cumbersome than doing the equivalent in COBOL. It also makes it very hard indeed to port an existing COBOL program and get exactly the same result in Java (so don't - use interop instead).

    BTW comp-1 is basically the same as float-short in MF but is a fixed type in ACU. MF support a compiler directive to make the MF compiler produce ACU style maths in this regard. Hence, it is much nicer to use float-short and float-long as there is no ambiguity as to what one indents - that is why I used them :)

    Best wishes - AJ

    ReplyDelete
  2. AFAIK, "Standard" COBOL doesn't allow for more than 18 digits of precision, so PIC S9(18)v(18) must be a vendor extension.

    Also, I believe all of the COMP-x usages are vendor-dependent, so you have to know your vendor and platform to truly understand what you're getting.

    Regards,

    Glenn

    ReplyDelete