Sort duplicates problem

[Howard Brazee]

We ran a job in the past with the following sort parms:
SORT FIELDS=(04,09,CH,A, SID
01,02,CH,A) INST
SUM FIELDS=NONE ELIMINATE DUPLICATES
DEBUG ABEND ABORT IF ERRORS
END

What I would like to do now is re-run this file and only select those duplicates that were eliminated the first time. (column 3 is different for the duplicates).

Is there a reliable way of getting only the ones that were eliminated in the previous run?

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

[Dean Montevago]

This will write the deleted dup records to the DD, don't know how to only
select them...

SUM FIELDS=NONE,XSUM

//SORTXSUM DD SYSOUT=X

-----Original Message-----
From: IBM Mainframe Discussion List [mailto:IBM-***@ibm-main.lst
Behalf Of Howard Brazee
Sent: Friday, May 21, 2004 12:46 PM
To: IBM-***@BAMA.UA.EDU
Subject: Sort duplicates problem


We ran a job in the past with the following sort parms:
SORT FIELDS=(04,09,CH,A, SID
01,02,CH,A) INST
SUM FIELDS=NONE ELIMINATE DUPLICATES
DEBUG ABEND ABORT IF ERRORS
END

What I would like to do now is re-run this file and only select those
duplicates that were eliminated the first time. (column 3 is different for
the duplicates).

Is there a reliable way of getting only the ones that were eliminated in the
previous run?

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

[Antonio Hacker]

Is there a possible problem in that, since EQUALS was not specified, they
may not sort the same way? Unlikely, I'd think...

Post by Dean Montevago
SUM FIELDS=NONE,XSUM

_________________________________________________________________
Best Restaurant Giveaway Ever! Vote for your favorites for a chance to win
$1 million! http://local.msn.com/special/giveaway.asp

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

[Howard Brazee]

Post by Antonio Hacker
Is there a possible problem in that, since EQUALS was not specified, they
may not sort the same way? Unlikely, I'd think...

Post by Dean Montevago
SUM FIELDS=NONE,XSUM

********************************* TOP OF DATA
****************************************************
ICE143I 0 BLOCKSET SORT TECHNIQUE SELECTED

ICE000I 1 - CONTROL STATEMENTS FOR 5740-SM1, DFSORT REL 14.0 - 12:02 ON FRI MAY
21, 2004 -
SORT FIELDS=(04,09,CH,A, SID

01,02,CH,A) INST

SUM FIELDS=NONE,XSUM PUT DUPS IN XSUM

$

ICE172A E XSUM IS NOT SUPPORTED - USE ICETOOL SELECT IF APPROPRIATE

DEBUG ABEND ABORT IF ERRORS

END

ICE052I 3 END OF DFSORT



I guess I'll try ICETOOL

[Swami, Shivprakash , GE Commercial Finance, NonGE]

If you have ICETOOL ,this should give you desired results.

//S1 EXEC PGM=ICETOOL
//TOOLMSG DD SYSOUT=*
//DFSMSG DD SYSOUT=*
//IN DD DSN=infile..
//LIST1 DD SYSOUT=*
//TOOLIN DD *
OCCUR FROM(IN) LIST(LIST1) ON(1,9,CH) ALLDUPS

Refer link below..

http://www.storage.ibm.com/software/sort/mvs/icetool/online/srtmtocc.html


-----Original Message-----
From: IBM Mainframe Discussion List [mailto:IBM-***@ibm-main.lst
Behalf Of Dean Montevago
Sent: Friday, May 21, 2004 12:51 PM
To: IBM-***@BAMA.UA.EDU
Subject: Re: Sort duplicates problem


This will write the deleted dup records to the DD, don't know how to only
select them...

SUM FIELDS=NONE,XSUM

//SORTXSUM DD SYSOUT=X

-----Original Message-----
From: IBM Mainframe Discussion List [mailto:IBM-***@ibm-main.lst
Behalf Of Howard Brazee
Sent: Friday, May 21, 2004 12:46 PM
To: IBM-***@BAMA.UA.EDU
Subject: Sort duplicates problem


We ran a job in the past with the following sort parms:
SORT FIELDS=(04,09,CH,A, SID
01,02,CH,A) INST
SUM FIELDS=NONE ELIMINATE DUPLICATES
DEBUG ABEND ABORT IF ERRORS
END

What I would like to do now is re-run this file and only select those
duplicates that were eliminated the first time. (column 3 is different for
the duplicates).

Is there a reliable way of getting only the ones that were eliminated in the
previous run?

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

[Howard Brazee]

Post by Swami, Shivprakash , GE Commercial Finance, NonGE
//S1 EXEC PGM=ICETOOL
//TOOLMSG DD SYSOUT=*
//DFSMSG DD SYSOUT=*
//IN DD DSN=infile..
//LIST1 DD SYSOUT=*
//TOOLIN DD *
OCCUR FROM(IN) LIST(LIST1) ON(1,9,CH) ALLDUPS
Refer link below..
http://www.storage.ibm.com/software/sort/mvs/icetool/online/srtmtocc.html

For some reason I keep having trouble getting a handle on icetool. That is a
nice start above, but I will need to modify it a bit, as I had a split-key:
SORT FIELDS=(04,09,CH,A, SID
01,02,CH,A) INST
SUM FIELDS=NONE ELIMINATE DUPLICATES
DEBUG ABEND ABORT IF ERRORS
END
*
/** 03,01,CH,A, EXT

The ESC field is the critical one - any fields after column 12 need to be
ignored.

[Frank Yaeger]

Post by Howard Brazee
SORT FIELDS=(04,09,CH,A, SID
01,02,CH,A) INST
SUM FIELDS=NONE ELIMINATE DUPLICATES
DEBUG ABEND ABORT IF ERRORS
END
What I would like to do now is re-run this file and only select those

duplicates that were eliminated the >first time. (column 3 is different
for the duplicates).

Post by Howard Brazee
Is there a reliable way of getting only the ones that were eliminated in the previous run?

SUM FIELDS=NONE keeps non-duplicates and the first record of each set of
duplicates.

You can accomplish the same thing using DFSORT/ICETOOL with:

SELECT FROM(IN) TO(OUT) ON(p,m,f,) FIRST

With SUM FIELDS=NONE, if EQUALS is in effect, the "first" record could be
any one of the duplicates whereas with EQUALS in effect the "first" record
is the first duplicate in the input file. SELECT always turns on EQUALS so
the "first" record is the first duplicate in the file.

If you want to keep the second and subsequent records of each set of
duplicates (that is, the ones that were eliminated by SUM FIELDS=NONE with
EQUALS or by SELECT with FIRST), you can use this DFSORT/ICETOOL statement:

SELECT FROM(IN) DISCARD(OUT) ON(p,m,f) FIRST

If you want to put the FIRST records in one data set and the non-FIRST
records in another data set, you can use this DFSORT/ICETOOL statement:

SELECT FROM(IN) TO(FIRST) DISCARD(NOTFIRST) ON(p,m,f,) FIRST

Frank Yaeger - DFSORT Team (IBM)
Specialties: ICETOOL, OUTFIL, Symbols, Migration
=> DFSORT/MVS is on the Web at http://www.ibm.com/storage/dfsort
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

[Howard Brazee]

---------- Forwarded Message Follows ----------

Path: news.colorado.edu!not-for-mail
From: "Howard Brazee" <***@ibm-main.lst>
Newsgroups: bit.listserv.ibm-main
Subject: Re: Sort duplicates problem
Date: Fri, 21 May 2004 18:13:52 GMT
Organization: University of Colorado, Boulder
Lines: 28
Message-ID: <c8lgsv$67s$***@peabody.colorado.edu>
References: <***@CINMLVEM09.e2k.ad.ge.com>
NNTP-Posting-Host: ums.cusys.edu
X-Trace: peabody.colorado.edu 1085163231 6396 192.12.240.129 (21 May 2004
18:13:51 GMT)
X-Complaints-To: ***@colorado.edu
NNTP-Posting-Date: 21 May 2004 18:13:51 GMT
X-Newsreader: News Rover 9.2.0 (http://www.NewsRover.com)
Xref: news.colorado.edu bit.listserv.ibm-main:222569

Post by Swami, Shivprakash , GE Commercial Finance, NonGE
//S1 EXEC PGM=ICETOOL
//TOOLMSG DD SYSOUT=*
//DFSMSG DD SYSOUT=*
//IN DD DSN=infile..
//LIST1 DD SYSOUT=*
//TOOLIN DD *
OCCUR FROM(IN) LIST(LIST1) ON(1,9,CH) ALLDUPS
Refer link below..
http://www.storage.ibm.com/software/sort/mvs/icetool/online/srtmtocc.html

For some reason I keep having trouble getting a handle on icetool. That is a
nice start above, but I will need to modify it a bit, as I had a split-key:
SORT FIELDS=(04,09,CH,A, SID
01,02,CH,A) INST
SUM FIELDS=NONE ELIMINATE DUPLICATES
DEBUG ABEND ABORT IF ERRORS
END
*
/** 03,01,CH,A, EXT

The ESC field is the critical one - any fields after column 12 need to be
ignored.

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

[Frank Yaeger]

Post by Howard Brazee
SORT FIELDS=(04,09,CH,A,
01,02,CH,A)

Note that you can use multiple ON fields as in these DFSORT/ICETOOL
statements:

SELECT FROM(IN) DISCARD(OUT) ON(4,9,CH) ON(1,2,CH) FIRST

SELECT copies the entire record.
OCCUR gives you a "report" of the duplicate values and counts.

For complete information on DFSORT's ICETOOL, see:

http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/ICE1CA00/6.0?DT=20031124143823

Frank Yaeger - DFSORT Team (IBM)
Specialties: ICETOOL, OUTFIL, Symbols, Migration
=> DFSORT/MVS is on the Web at http://www.ibm.com/storage/dfsort

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

[Howard Brazee]

---------- Forwarded Message Follows ----------

Path: news.colorado.edu!not-for-mail
From: "Howard Brazee" <***@ibm-main.lst>
Newsgroups: bit.listserv.ibm-main
Subject: Re: Sort duplicates problem
Date: Fri, 21 May 2004 18:04:45 GMT
Organization: University of Colorado, Boulder
Lines: 33
Message-ID: <c8lgbs$5nh$***@peabody.colorado.edu>
References: <BAY17-***@hotmail.com>
NNTP-Posting-Host: ums.cusys.edu
X-Trace: peabody.colorado.edu 1085162684 5873 192.12.240.129 (21 May 2004
18:04:44 GMT)
X-Complaints-To: ***@colorado.edu
NNTP-Posting-Date: 21 May 2004 18:04:44 GMT
X-Newsreader: News Rover 9.2.0 (http://www.NewsRover.com)
Xref: news.colorado.edu bit.listserv.ibm-main:222567

Post by Antonio Hacker
Is there a possible problem in that, since EQUALS was not specified, they
may not sort the same way? Unlikely, I'd think...

Post by Dean Montevago
SUM FIELDS=NONE,XSUM

********************************* TOP OF DATA
****************************************************
ICE143I 0 BLOCKSET SORT TECHNIQUE SELECTED

ICE000I 1 - CONTROL STATEMENTS FOR 5740-SM1, DFSORT REL 14.0 - 12:02 ON FRI MAY
21, 2004 -
SORT FIELDS=(04,09,CH,A, SID

01,02,CH,A) INST

SUM FIELDS=NONE,XSUM PUT DUPS IN XSUM

$

ICE172A E XSUM IS NOT SUPPORTED - USE ICETOOL SELECT IF APPROPRIATE

DEBUG ABEND ABORT IF ERRORS

END

ICE052I 3 END OF DFSORT



I guess I'll try ICETOOL

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

[Frank Yaeger]

Post by Howard Brazee
SUM FIELDS=NONE,XSUM PUT DUPS IN XSUM
$
ICE172A E XSUM IS NOT SUPPORTED - USE ICETOOL SELECT IF APPROPRIATE

For more information on using DFSORT/ICETOOL's equivalent of XSUM, see:

http://www.storage.ibm.com/software/sort/mvs/tricks/srtmst01.html#t12

Frank Yaeger - DFSORT Team (IBM)
Specialties: ICETOOL, OUTFIL, Symbols, Migration
=> DFSORT/MVS is on the Web at http://www.ibm.com/storage/dfsort

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

[Howard Brazee]

Post by Frank Yaeger
Note that you can use multiple ON fields as in these DFSORT/ICETOOL
SELECT FROM(IN) DISCARD(OUT) ON(4,9,CH) ON(1,2,CH) FIRST
SELECT copies the entire record.
OCCUR gives you a "report" of the duplicate values and counts.

That worked. I saw multiple OUT parms in the documentation, but somehow my
brain and ICETOOL's documentation tend to work differently. (I usually give up
and go back to old fashioned sort - what I know of both utilities is much more
than other people at this shop). So I experimented unsuccessfully.

Thanks.

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html