Variable Length Record for an Entry-Sequenced File

[Jason Paradiso]

Hey, there, HPNonstoppers,

I'm in the process of conducting analysis for a project that requires to create a transaction log file to capture OLTP transactions. I read the eCOBOL manual and it suggested that an entry-sequenced file will be a good candidate for transaction logging file that stores data chronologically.

Our HPNS runs OS with J06.22+ that allows a max record length of 27576 bytes for an entry-sequenced file and I need to record size around 10k bytes on a Format 2. During our team brainstorming, someone asked if it would be better off to create variable length for this new transaction log.

I'm not quite familiar with how a cobol program would access a variable length entry-sequenced record so I did go through ecobol manual trying to find some info, but it did not completely answer all my questions. So, here I am with some of my unanswered questions:

1). From the ecobol manual, I understand that I'll need to include either a RECORD CONTAINS rec-1 TO rec-2 CHARACTERS or a RECORD IS VARYING IN SIZE FROM rec-1 TO rec-2 CHARACTERS clause, in the FD entry to declare a variable length record. Nevertheless, the file schema would still be set up with a fixed record length for the entry-sequenced file, say 12k bytes. How does declaring a variable length record for a fixed length record entry-sequenced file gonna buy us anything? Does it imply the fixed record length of 12k would potentially store more than 1 record based on the variable length (e.g., 2 5k variable length records within 12k length)?

2). What would be the best way to estimate/determine the ideal record size to prevent block split and minimize the slack space? In the ecobol manual, it mentioned that when the run-time routines execute a REWRITE or WRITE statement referencing a file of variable-length records, the number of character positions in the record written is determined. Does that mean it would always cause block split if the program declares variable length for an entry-sequence file with a fixed record length (say, 12k bytes)?

3). Last but not the least, I guess the question I kept asking myself is - given an entry-sequenced file schema that defines a fixed length of record size (e.g., 12k), how would the records of variable length written into a block?

Thanks for your attention to my questions and any feedback or suggestion will be greatly appreciated!

Jason

[Randall]

Post by Jason Paradiso
Hey, there, HPNonstoppers,
I'm in the process of conducting analysis for a project that requires to create a transaction log file to capture OLTP transactions. I read the eCOBOL manual and it suggested that an entry-sequenced file will be a good candidate for transaction logging file that stores data chronologically.
Our HPNS runs OS with J06.22+ that allows a max record length of 27576 bytes for an entry-sequenced file and I need to record size around 10k bytes on a Format 2. During our team brainstorming, someone asked if it would be better off to create variable length for this new transaction log.
1). From the ecobol manual, I understand that I'll need to include either a RECORD CONTAINS rec-1 TO rec-2 CHARACTERS or a RECORD IS VARYING IN SIZE FROM rec-1 TO rec-2 CHARACTERS clause, in the FD entry to declare a variable length record. Nevertheless, the file schema would still be set up with a fixed record length for the entry-sequenced file, say 12k bytes. How does declaring a variable length record for a fixed length record entry-sequenced file gonna buy us anything? Does it imply the fixed record length of 12k would potentially store more than 1 record based on the variable length (e.g., 2 5k variable length records within 12k length)?
2). What would be the best way to estimate/determine the ideal record size to prevent block split and minimize the slack space? In the ecobol manual, it mentioned that when the run-time routines execute a REWRITE or WRITE statement referencing a file of variable-length records, the number of character positions in the record written is determined. Does that mean it would always cause block split if the program declares variable length for an entry-sequence file with a fixed record length (say, 12k bytes)?
3). Last but not the least, I guess the question I kept asking myself is - given an entry-sequenced file schema that defines a fixed length of record size (e.g., 12k), how would the records of variable length written into a block?
Thanks for your attention to my questions and any feedback or suggestion will be greatly appreciated!
Jason

I don't think the eCOBOL manual is the best place to look. There are other manuals - File Utilities, Guardian Programmer's Guide, Proc Calls, that have more details. FILE_WRITEX64_ is the call to examine.

As far as block splits, they are rare in entry-sequenced files where everything is effectively variable length whether you intend it or not. The more relevant question is will your log entries fit into a block or not. That is a min/max optimization question on minimizing dslack. What happens is you have slack, which you want to be small in a log file because you are not going to have subsequent inserts. A block split is more relevant in key sequenced files where a block can split either when an index causes a split, or when a rewrite can force the split.

I'm sure others have more info on this.

[Bill Honaker]

Post by Jason Paradiso
Hey, there, HPNonstoppers,
I'm in the process of conducting analysis for a project that requires to create a transaction log file to capture OLTP transactions. I read the eCOBOL manual and it suggested that an entry-sequenced file will be a good candidate for transaction logging file that stores data chronologically.
Our HPNS runs OS with J06.22+ that allows a max record length of 27576 bytes for an entry-sequenced file and I need to record size around 10k bytes on a Format 2. During our team brainstorming, someone asked if it would be better off to create variable length for this new transaction log.
1). From the ecobol manual, I understand that I'll need to include either a RECORD CONTAINS rec-1 TO rec-2 CHARACTERS or a RECORD IS VARYING IN SIZE FROM rec-1 TO rec-2 CHARACTERS clause, in the FD entry to declare a variable length record. Nevertheless, the file schema would still be set up with a fixed record length for the entry-sequenced file, say 12k bytes. How does declaring a variable length record for a fixed length record entry-sequenced file gonna buy us anything? Does it imply the fixed record length of 12k would potentially store more than 1 record based on the variable length (e.g., 2 5k variable length records within 12k length)?
2). What would be the best way to estimate/determine the ideal record size to prevent block split and minimize the slack space? In the ecobol manual, it mentioned that when the run-time routines execute a REWRITE or WRITE statement referencing a file of variable-length records, the number of character positions in the record written is determined. Does that mean it would always cause block split if the program declares variable length for an entry-sequence file with a fixed record length (say, 12k bytes)?
3). Last but not the least, I guess the question I kept asking myself is - given an entry-sequenced file schema that defines a fixed length of record size (e.g., 12k), how would the records of variable length written into a block?
Thanks for your attention to my questions and any feedback or suggestion will be greatly appreciated!
Jason

Hi, Jason,

Randall's post is correct with regards to block splits.

DDL has no concept of variable-length structures. It will generate fixed-length structures, and any FUP CREATE statements will have REC and BLOCK attributes set.

COBOL will write the data based on the FD. You are looking for the VARYING phrase on page 172 of the NonStop COBOL manual in the Data division section:

RECORD VARYING FROM n1 TO n2 CHARACTERS DEPENDING ON len-var.

n1 and n2 are literals defining the smallest and largest lengths. len-var is the name of an unsigned integer data item.

When writing variable length, you must set the len-var prior to the WRITE. Typically you would use the STRING verb and WITH POINTER option to know how long to write.
Alternatively, you might have to use a PERFORM loop looking from the end of a buffer for the last trailing blank and calculate the length.

When reading, you must READ and then check the value of len-var. You should use the len-var value in a reference modifier:

READ var-file INTO var-rec.
MOVE var-rec (1:len-var) TO tareget-buffer.

You can REWRITE an entry sequenced record but it's not usually done in Logging. You can't DELETE a record (altough you can REWRITE it with a zero-byte count).

As for your third (blocking) question, you should read look in the Enscribe Programmers manual and find "Figure 31 Block Format for Structured Format 2 Files".

It's not a simple case of '10 5,000 byte records fit in a 50,000 byte block'.

"Table 7 Comparison of Format 1 Versus Format 2 Files" tells you some maximums, but in the case of variable-length records you'd have to know statistics.
Which you won't know until th app has run for a while. You can just let the Disk Process manage it and it should be fine.

Good luck,
Bill