Sort Question [AD - Kind of]

[Blaicher, Christopher Y.]

Syncsort's MFX sort when using the Global Sort Monitor measures systems resources, both on an instantaneous basis and on a historical basis to judge how much memory each sort should use. If you give a MFX sort a REGION=0M it may use 8M or 1.5G depending on what the current memory demands are by other processes, and what the historical data indicates will be needed over the life of the sort. MFX automatically does all that for you.

If you have a 10G machine and over 2G of it are free, who cares that a sort comes along and uses a gig. It isn't hurting anything. On the other hand if there is paging already happening, you only want the sort to take as much as is absolutely necessary to do the job with relative efficiency.

Chris Blaicher
Technical Architect
Mainframe Development
P: 201-930-8234 | M: 512-627-3803
E: ***@syncsort.com

Syncsort Incorporated
2 Blue Hill Plaza #1563
Pearl River, NY 10965
www.syncsort.com

Data quality leader Trillium Software is now a part of Syncsort.


-----Original Message-----
From: IBM Mainframe Discussion List [mailto:IBM-***@LISTSERV.UA.EDU] On Behalf Of Farley, Peter x23353
Sent: Monday, October 2, 2017 12:49 PM
To: IBM-***@LISTSERV.UA.EDU
Subject: Re: Sort Question

-----Original Message-----
On Behalf Of Paul Gilmartin
Sent: Monday, October 02, 2017 12:24 PM
Subject: Re: Sort Question

With respect to the REGION size, remember that SORT (both IBM and its competitors) is most efficient the more memory you can let him have. I have found this especially true for COBOL-based SORT's.

But does this contend adversely with concurrent jobs?

IMHO it is the job of the Capacity and Performance team to tell the application programming teams when application jobs are contending for memory and causing paging, especially if it causes SLA's to be missed or near-missed.

Consider using a region of at least 500M for large-volume sorts, and be sure to tell SORT he can use (most of) it via the SORT parameters. The more memory he has available the fewer SORTWK's and SORTWK I/O's he will need to use.

REGION=0K?

Only if allowed by IEFUSI. Many large shops do not allow it except with special dispensation and approval, specifically because of the issue you raised above about memory contention.

Also "buffer up" all of your high-volume input files with high BUFNO (QSAM) or AMP='BUFND=XXXX,BUFNI=YYYY,RMODE31=ALL' (VSAM) parameters (as appropriate). Consider using SUBSYS=BLSR or SMB parameters for high-volume VSAM inputs as well.

I would hope (wish) that DFSORT would supply optimum defaults. But can an application discern in OPEN exit whether the options were supplied in JCL or as OS defaults?
Does DFSORT rely on QSAM or on idiosyncratic EXCP? I'd expect that in the era of oscillating merge it relied on EXCP.

Of course SORT uses his own EXCP (or OCO media manager interfaces) where possible, as it is far more efficient than QSAM. I was referring to the OP's original COBOL source for this SORT question. My reply concerned application-read input files, not directly-read-by-SORT files. For JCL sorts I always let SORT decide how to optimize his files, but COBOL SORT's act as E15 and/or E35 exits, so COBOL I/O can and often is being used for those I/O's, whether directly for the file to be sorted or a multiplicity of inputs used to construct the SORT input and output records.

Memory is (relatively) inexpensive, so use it to your advantage. Your SLA's will appreciate it.

Peter
--

This message and any attachments are intended only for the use of the addressee and may contain information that is privileged and confidential. If the reader of the message is not the intended recipient or an authorized representative of the intended recipient, you are hereby notified that any dissemination of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail and delete the message and any attachments from your system.

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions, send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

________________________________



ATTENTION: -----

The information contained in this message (including any files transmitted with this message) may contain proprietary, trade secret or other confidential and/or legally privileged information. Any pricing information contained in this message or in any files transmitted with this message is always confidential and cannot be shared with any third parties without prior written approval from Syncsort. This message is intended to be read only by the individual or entity to whom it is addressed or by their designee. If the reader of this message is not the intended recipient, you are on notice that any use, disclosure, copying or distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please immediately notify the sender and/or Syncsort and destroy all copies of this message in your possession, custody or control.

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

[Elardus Engelbrecht]

Post by Blaicher, Christopher Y.
Syncsort's MFX sort when using the Global Sort Monitor measures systems resources, both on an instantaneous basis and on a historical basis to judge how much memory each sort should use.

I wish with a really sore heart that all applications and compiled programs do those two tricks automagically and in a jiffy.

... like that windoze operating system which remembers your few last used application programs and store it in the memory until the page file(s) gets full... You run that application again, those associated programs are already in the memory, no need to read in from your hard disk.

On the PCs, these languages, Clarion and Clipper, have background things which dynamically optimize your memory and files in use while the MSDOS machines are idle... Nice features which proved useful.

I believe on C64 from Commodore does the same thing, it does that infamous garbage collection when your toy is idle...

So, something which is used to adjust your own performance while running, yes, there is some overhead with those dynamic measurements, but if overall CPU usage and I/O usage are fewer, that should be good.

I know about Linklist, COFVLFxx, XCF structures for RACF and such animals, but they are somewhat static ('historical basis'), not dynamic like that SyncSort Feature.

Post by Blaicher, Christopher Y.
On the other hand if there is paging already happening, you only want the sort to take as much as is absolutely necessary to do the job with relative efficiency.

That is another feature I want to steal.... Sorry, but is it Friday today? ;-)

Thanks for your notes. Much appreciated.

Groete / Greetings
Elardus Engelbrecht

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN