Declarative Sorting Of Dynamically Allocated In-Memory Tables With COBOL

One of the things COBOL is really good at is data processing.

Its declarative data model makes it second only to SQL for this purpose. Both in Managed COBOL and native COBOL, Micro Focus COBOL can sort an in-memory table just by saying 'sort'. Not only that, a little trick with pointers and the linkage section means that we can sort any sized table and allocate the memory for that table on the fly. Here I am using Visual COBOL (Visual Studio COBOL) to work through these ideas. To do this, I am using an imaginary example of a retail data control system where we have product variants and stock keeping units (skus). Our little program shows how we can sort variant name/sku pairs by the sku number.

What does declarative mean?

Here we simply declare the structure of our table and how it is indexed and can be sorted:

linkage section.
       01 variants occurs 1 to 1000 depending on variant-count
                   ascending key is sku
                   indexed   by jump-start-index.
           03 sku  binary-long.
           03 decr pic x(20).

Because we have put this in the linkage section, we have declared all this but we have not allocated any storage to it yet - that comes later. Now we sort:

sort variants

Now that is the bit I love! Having declared the data and key structure of the table, COBOL knows everything it needs to know about how to sort it. We do not have to call some sort function passing in some functional thing like a lambda or a call back. All that complexity is taken away. We want the table sorted - so we tell COBOL to sort it! If only my kids were so easy to instruct...

How About Pointers And Allocation?

As mentioned, the declaration of the table is in the linkage section. I have also created a working storage item 'mem-ptr' which is usage pointer. This approach means we can use CBL_ALLOC_MEM to assign a block of memory to use for storing our table. The size of the block to allocate is worked out from the size of each record via the "length of" keyword pair and multiplying the result by variant-count. The location of that memory is pointed to by mem-ptr and we can tell COBOL to use it for our table by the statement "set address of variants(1) to mem-ptr". However, COBOL does need to know how big the table is. This is done by using variant-count in the "depending on" clause of the table declaration.

Putting It Together In Visual COBOL

First open Visual Studio:

Then start a new project:

Then pick a native COBOL template:

Take the created source and replace it with the source below:

$set sourceformat(variable)
       program-id. "sorter".

       working-storage section.
       01 mem-ln        binary-long.
       01 mem-flags     pic x(4) comp-5 value 0.
       01 int-bl        binary-long.
       01 mem-ptr       usage pointer.
       01 status-code   pic x(2) comp-5.
       01 variant-count binary-long.
       
       linkage section.
       01 variants occurs 1 to 1000 depending on variant-count
                   ascending key is sku
                   indexed   by jump-start-index.
           03 sku  binary-long.
           03 decr pic x(20).
           
       procedure division.
       
           move 10 to variant-count
       
           set mem-ln to length of variants
           multiply mem-ln by variant-count giving mem-ln
           
           call "CBL_ALLOC_MEM" 
                using     mem-ptr
                by value  mem-ln mem-flags
                returning status-code
           if not status-code = 0
               display    "Failed to get memory"
               stop run
           end-if
           
           set address of variants(1) to mem-ptr
           
           move 1234       to  sku(1)
           move "beans"    to decr(1)
           move 12         to  sku(2)
           move "fish"     to decr(2)
           move 4532       to  sku(3)
           move "cat food" to decr(3)
           move 2342       to  sku(4)
           move "dog food" to decr(4)
           move 1231       to  sku(5)
           move "sauce"    to decr(5)
           move 1254       to  sku(6)
           move "bread"    to decr(6)
           move  999       to  sku(7)
           move "chicken"  to decr(7)
           move    1       to  sku(8)
           move "beer"     to decr(8)
           move    2       to  sku(9)
           move "wine"     to decr(9)
           move  9999      to  sku(10)
           move "tnt"      to decr(10)
           
           sort variants
           
           perform varying int-bl from 1 by 1 until int-bl greater variant-count
               display sku(int-bl) " =- " decr(int-bl)
           end-perform
           goback
           .
       end program "sorter".

Giving this:

Now click in the left margin to put a break point on the "goback":

Now step into the program using the debugger and it will stop on the break point:

We can now see the sorted output!

For more stuff like this - check out the Visual COBOL Knol and The Managed COBOL Knol community sites

Why APM Is Important To COBOL?

Some of you may be asking "why is APM relevant?".  What has application portfolio management got to do with COBOL?  And how does it relate to everything else I've read on this blog?

Let me connect the dots.  (forgive me if I wander a bit *smile*)

To many folks, COBOL is "old code" that has got to go.  The reasons many provide usually have something to do with the user interface or the data, but not the core functionality these applications provide.  Those using the applications on a daily basis hate the old character interfaces.  This is thanks to the fact that everything else they use has a slick web interface and they can generate their own adhoc reports without waiting for weeks or months for IT to get around to it.

To quote a CIO I spoke with recently "Users hate using the character screens but we couldn't do business without the functionality the systems provide".  He also had issues with the data.  Since everything was "locked" into the applications, they had to create a data warehouse which replicated the various information and made it more accessible to the business users.

He really really really would like to do something with the applications, but he didn't buy into the "let's rewrite it in (your language here)" pitch.  This is due to the fact that he realizes those applications are very complex and rewriting them introduces unacceptable risk.  If he understood how these applications worked, he might be able to do something with them.

I saw another example of this today while visiting one of the largest insurance companies in America.  One team within the company was tasked with trimming $100 Million out of the company's operation budget.  To do so, they started asking questions about their mainframe-based COBOL applications.  What they found was that no one understood everything these applications did.  Nor did they know how interwoven the various applications were with the rest of their infrastructure.  Sure they knew pieces and parts, but no one had the big picture.

Just to make sure you didn't miss my point, no one in this multi-billion dollar company understands how their systems work or what they consist of.

And we are talking a company with literally hundreds of architects, project managers, application developers, etc.  Even those who have been with the company 30 years only knew what parts of the various application elements did or were comprised of.  And these folks are making multi-million dollar decisions based on what they know about their systems every day!

Yikes!

I've digressed a bit, but hang in there, I can draw it all together. I think... *wink*

How would both of my examples solve their problems?  APM.

With an APM-type solution, that old code becomes a reusable building block for the new applications your company needs.  By using an APM solution to gain an understanding of the application and its composition, you could then take that "old COBOL code" and re-purpose it with a new user interface or by opening up the data layer. 

With an APM solution, companies can see how to reduce their IT expenses by identifying dead application elements, unnecessary application complexity, duplicate functionality and data.  APM-based solutions can provide management a real-time picture of these systems.  Who knows, they might even realize how risky a complete rewrite to (your language here) really is.

Recall the earlier posts on Visual COBOL? (see how I wove that in there?  slick huh?)

Once you have identified those pieces which are key to the business, you could use a tool like Micro Focus Visual COBOL to create a COBOL-based web service.  Then those who are building the new user interfaces could use whatever language they want to build that slick new user interface.  And by doing so, COBOL continues to have a place in the environment.

Or what about the post on migration?  (yes that was another post I made) With an APM solution, you can identify what pieces of an application can be moved off the mainframe to a Windows or UNIX or Linux server.

So, back to my point.  I believe that APM-based technologies are critical to the future of COBOL. 

They can give new life to those COBOL systems...

That's my thinking anyways.  What's yours?

Application Portfolio Management - Best Practices

It's tough to control your application portfolio. Your systems have been developed over the course of years or even decades. The people that developed the apps may have moved on to other roles and documentation is out of date. So, the portfolio gets more and more complex. And that complexity means they are slower to adapt and more expensive to maintain.

Application portfolio management is an approach to help return control to application managers. In this series of posts, I'll take a look at best practices for managing the application portfolio.

• Goal: Constant fire-fighting is no way to run a development organization. Especially in today's era of tight budgets and fast change. In this post I'll summarize the goal of APM and set the stage for a discussion of best practices. Read the post.
  
  
• Questions and Metrics: APM data should answer questions that address a specific goal. Say, ‘why is this business process inflexible?’, ‘where can I cut costs?’, or ‘where is my software architecture flawed?’. To answer these different questions requires different combinations and weightings of data (user surveys, application code, or external sources). Sometimes more of one source, sometimes another. Read the post.
  

• Decision-Makers: APM data needs vary based on where you are in the organization. Higher level managers require higher level abstractions, particularly of technical metrics. Also, different types of users will have different data needs. An architect may want technical complexity data, but it may only be meaningful to him if it is filtered by architectural models. Read the post.
  

• Maturity: There are different levels of maturity for decision-making. This maturity directly affects which metrics are accessible in the first place and also indirectly because it determines the kind of business goals that an organization is prepared to address. Read the post.
  

• What’s next: As a particular initiative moves from “decision” to “action”, different data may be needed. More “bottom-up” data may be necessary to implement the decisions at this stage. Further, different metrics can be monitored to ensure the success of a given development or modernization project as it is executed. Read the post.

Precision: What is it Precisely?

Alex has an interesting post over on Nerds Central which I believe you may find of value.

He details a couple of useful items specific to decimal precision in Cobol and Java.  It is interesting  because it illustrates the use of a Cobol data type I've not played with, float-long. 

Float-Long came about a few years ago, along with Float-Short and Float-Extended and was made part of the 2002 Standard I believe.  To make a long story short, Float-Long isn't quite as accurate as you would hope, but does have its uses.  Alex provides a complete example showing how to acheive both an exact arithmatic answer using a Comp data type and how to use the Float-Long data type.

The other reason I find the post of interest is that in his article, it appears the Java code can only duplicate what the Cobol Float-Long data type provides (which is inaccurate), not the more precise answer he achieved using the more standard Cobol data type of Comp-1. 

Am I mistaken in my interpretation of the Java sample he provided?

This looks like a red flag to me if your Java code is doing any math which requires a high level of accuracy on the right side of the decimal.

Can someone verify this and post a Java-based example confirming/denying the problem?

I'm curious but only know enough Java to be able to nod in the right places when talking with a Java developer. *smile*

P.S.:  The Visual Studio 2010 Launch in Las Vegas was quite the event.  Thanks to various issues with my plane ride (rerouted due to medical emergency and then detained in Albuquerque, NM due to mechanical issues, etc), it only took me 20 hours or so to make the 3 and 1/2 hour trip home.  At least I wasn't flying to London!

Visual Cobol. You gotta see it

Hey folks!

I'm in lovely Las Vegas on the conference floor talking with customers about the new Visual Cobol release which is being announced at the Visual Studio 2010 Launch.

So far the feedback has been that this is really slick stuff indeed.  This is the first time I've gotten a chance to look at the new syntax support.  Micro Focus development has done a tremendous job simplifying the syntax.  If you haven't looked at the articles out on C# Corner, go take a quick look.

Oh well, gotta go.  Mike wants his laptop back!

JCL and Procs Running on a Windows Server Near You!

Hey folks,

Sorry to be slow in posting anything on mainframe migrations that I promised. 

Seven weeks ago I was grabbed by the neck and tossed into shark infested waters on a pilot project for a large retailer.  They were looking into the idea of turning off their mainframe.  For some reason or another they didn't just plop down their checkbook and write out a check.  "Run our application on a Windows?  Prove it." they said. 

So after many fun days and nights of work we demonstrated their application running off the mainframe.  Nothing like reinventing an environment which took twenty years to setup in a handful of weeks.

Looking back, we spent almost all of our time building out the plumbing.  Getting three CICS regions stood up that matched the mainframe, building out their PDS, Loadlib and Joblib structures on the server, wiring in an ISC link back to the mainframe, setting up Ezasockets, etc., took most of the time.  The application elements (BMS, JCL, PROCs, COBOL, etc) came down with no changes.  And once we finished the wiring, those elements ran just fine after compilation and whatnot.

In the past, the approach would have been to convert the JCL to a VB Script.  But nowadays JCL can be run pretty much as is.  The biggest caveats are around third party tools such as FileAid, Easytrieve, and so on and so forth.  Alternate approaches to those elements may have to be put together before the jobs can run.  But the core elements can run just like they are.

For instance, consider the sample JCL below:

//JCLTEST JOB 'JCL TEST',CLASS=B,MSGCLASS=A

//*
//* DELETE EXISTING DATASETS
//*
//GETRID EXEC PGM=IDCAMS
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
DELETE MFIJCL.OUTFILE.DATA
SET MAXCC=0
//*
//* ALLOCATE AND WRITE RECORDS TO A DATASET FROM A USER PROGRAM
//*
//CREATE EXEC PGM=JCLCREAT
//OUTFILE DD DSN=MFIJCL.OUTFILE.DATA,DISP=(,CATLG),
// DCB=(LRECL=80,RECFM=FB,DSORG=PS),
// SPACE=(800,(10,10)),UNIT=SYSDA
//SYSOUT DD SYSOUT=A
//*
//* USE THE IEBGENER SYSTEM UTILITY TO COPY RECORDS INTO A TEMPORARY
//* DATASET AND PASS ON FOR USE BY SUBSEQUENT STEPS
//*
//GENER EXEC PGM=IEBGENER
//SYSPRINT DD SYSOUT=*
//SYSUT1 DD *
Rec 1
Rec 2
Rec 3
Rec 4
Rec 5
//SYSUT2 DD DSN=&&JTEMP,DISP=(,PASS),
// DCB=(LRECL=80,RECFM=FB,DSORG=PS),
// SPACE=(800,(10,10)),UNIT=SYSDA
//*
//* READ RECORDS FROM THE TEMPORARY DATASET CREATED IN THE PREVIOUS
//* STEP
//*
//READ EXEC PGM=JCLREAD
//INFILE DD DSN=&JTEMP,DISP=(OLD,DELETE)
//SYSOUT DD SYSOUT=A
//

It runs just fine on a Windows or UNIX box using the Micro Focus tools.  And before you ask, yes GDG's and restart logic work too.  My sample just didn't have any in it and I didn't go looking for a better sample.  You'll have to use your imagination.

The hardest part for the project involved getting the customer's CICS user exits brought over and working.  I had to wire in ISC links back up to the mainframe CICS region to execute routines not included in the downloaded project components.  But I digress... *smile*

I was talking JCL and Procs.  I have yet to find JCL that runs on z/OS which won't work... Those things I have that don't work are the utilities or third party tools, not bone-stock JCL.  Does anyone have a specific question on migrating JCL that they are dying to ask?

Are you 'Visual'?

If you're like a lot people you learn more by observing how a process or task is completed than by having it explained or reading a book. While manuals and instructions are important, having hands-on experience with something reinforces the training being received and really 'drives it home'. I am one of the visual types myself. I like to work with a new process or product, dig into it, see what it's made of, try to push it to its limit and see what happens. This past week I had a wonderful time in Newbury, England with the Micro Focus Development Teams kicking the tires on Visual COBOL.

"Visual COBOL"?!?!? Yep...Visual COBOL. Now this isn't some marketing campaign kicked off by the Micro Focus Execs to try and sell more re-branded, same old run of the mill compilers (trust me, this ain't that at all). This is a whole new product that was developed to take advantage of the Microsoft .NET environment and platform. Before I explain about Visual COBOL I need to fill in some details. On or about April 12th Microsoft is going to be releasing Visual Studio 2010. This is going to be a major release for Microsoft, along with several other products. There will also be a new version of the .NET Framework being released. As part of the Global Agreement between Microsoft and Micro Focus, Micro Focus is required to release a product that will take advantage of the new architecture and features of the Visual Studio 2010 product... and do it within 30 days of the VS2010 launch date. Enter Visual COBOL.

Visual COBOL is 'not the same old COBOL' from Micro Focus. Not even close. Oh there are similarities; like a 30 plus year history of COBOL development experience, a lineage that has spanned multiple platforms, multiple language revisions, and dialects and has had more developers banging away on it than most other language development environments. But that's where the similarity ends. Visual COBOL is a whole new development product. The folks in Newbury took the knowledge and experience they have, married it with the excellent technology of .NET and created a whole new compiler and runtime environment. As some people have said in the past about COBOLs' growth..."this ain't your Daddy's COBOL" and they are so right. Now don't assume just because it's a "whole new development product" that your old COBOL code won't work on it...wrong answer Skippy! Although the development product is brand new, Micro Focus has taken great care to ensure backward compatibility. To that end, back to this week!

The week in Newbury was designed to bring a group of people from outside Newbury, along with the Newbury and Sophia Development Teams together to really kick the tires on Visual COBOL. The goal was to see where the product was in relation to the project timeline, what issues were noted, and what could be resolved before the go live date. Everyone brought along samples of code and the first thing everyone did was try out their code on the compiler. I can attest that 100% of my previous code compiled and executed. The Team from Japan was also very impressed that not only did their code execute, but the interface was accurate with very few issues noted. The Team from France had similar results. The major portion of the week though was devoted to a process called 'STX' or Super Test eXtended. STX was a three day programming marathon session by everyone involved. All development on Visual COBOL was halted. The Development teams were split into teams of two and each was to come up with a project using Visual COBOL. They were supposed to use Visual COBOL as you or I would, as an end-user. All coding was to be done in COBOL, only COBOL. This was the first time I participated in an exercise such as this. To put it mildly, I felt like a fish out of water. While I like to think I'm a good coder I felt like a kid out of elementary school around these people. The way their minds think and the things they came up with were amazing. And their ability to go from concept to prototype in a very short period of time was awesome. The end result...WOW!

Friday was our presentation day, the day we regrouped and showed off what we had come up with and describe our experiences using Visual COBOL. Now I need to mention there were developers who were new to Micro Focus that were used to coding in Java, C++, in Eclipse, on Unix/Linux who were involved. The bottom line though was they had to use Visual COBOL running under Visual Studio 2010. The end results were pretty amazing. (As an editorial note, Micro Focus has some awesome developers. The stuff these guys not only came up with but demonstrated was amazing. Back to the article).

One of the Development Directors (hey, he was a coder at one time before he became a 'Director') came up with a copyfile parsing routine. By supplying a copybook the routine would identify different segments of the copybook, identifying keys, redefines, etc. and apply different colorizations to the fields depending on usage. Here's a screen shot.

Another example was the snake game. Using the Micro Focus ADIS controls (going back to the bit about backward compatibility here) a snake game was created. A snake game is where you have to move around the screen eating letters or numbers but not over-step your tail as the snake grows in length as you eat the letters off the screen. The Team in Sophia, Bulgaria created an XML parser/generator that could read in a file and generate XML that could be used in a cloud-computing environment. Another person created a WPF tic-tac-toe game. One team of 2 even went so far as to search the internet for an interesting C# sample to see if they could take what was done in C# and convert it to Visual COBOL. (I found this example as well but am nowhere near as savvy as these guys are so didn't even attempt it). Have you ever played Visual COBOL Tetris? This version looked awesome. Great colors, quick response time, smooth graphics. All built using Visual COBOL and taking advantage of the .NET Framework 4.0. I'm hoping this example makes it into the release. It was really cool and to be honest, looked better than the C# version (sorry Microsoft). There were other examples but these were the ones presented to the excited crowd.

Did you notice one thing about the examples described? All but one were games! That's right...games. COBOL is no longer relegated to only crunching business information and generating reports. While it is still very well suited for that task it has come so much farther than that, thanks to companies like Microsoft and Micro Focus. The agreement between these two companies it translating into a lot of opportunity for companies with existing COBOL resources to become more profitable with what they have. By taking their existing COBOL source code and development teams, spending some time and money on retraining, customers will be able to realize some huge technological advances for what turns out to be little investment. Visual COBOL and Visual Studio 2010 will greatly enable customers to leap forward technically and afford them the ability to please their customes.

I'm really excited to have been a part of this effort and am very much looking forward to the release of Visual Studio 2010 and Visual COBOL. As soon as I get the OK I'll start describing the changes that are coming, not only in presentation but in the compiler. Trust me, it will make our COBOL code even easier to read and follow for non-COBOL programmers who can then learn how to code COBOL to REALLY become productive!

Happy Coding!