#!/usr/bin/perl

#
# dbcolpercentile.pm
# Copyright (C) 1997-2024 by John Heidemann <johnh@isi.edu>
#
# This program is distributed under terms of the GNU general
# public license, version 2.  See the file COPYING
# in $dblibdir for details.
#


=head1 NAME

dbcolpercentile - compute percentiles or ranks for an existing numeric column

=head1 SYNOPSIS

    dbcolpercentile [-rplhS] [--mode MODE] [--value WEIGHT_COL] column

=head1 DESCRIPTION

Compute a percentile, ranking, or weighted percentile of a column of numbers.
The new column will be called I<percentile:d> or I<rank:q> or I<weighted:d>
depending on the mode.

Ordering is given by the specifed column.

In weighted mode, by default the same column as ordering is used for weighting.
Alternatively, give a different column for weighting with C<-v>.

Non-numeric values are ignored.

If the data is pre-sorted and only a rank is requested,
no extra storage is required.
In all other cases, a full copy of data is buffered on disk.
Output will be sorted by COLUMN.

=head1 OPTIONS

=over 4

=item B<-p> or B<--percentile> or B<--mode percentile>

Show percentile (default).
Percentile is the fraction of the cumulative values at or lower than the current value, relative to the total count.

=item B<-P> or B<--rank> or B<--nopercentile> or B<--mode rank>

Compute ranks instead of percentiles.

=item B<-w WEIGHT_COL> or B<--weighted WEIGHT_COL> or B<--mode weighted>

Compute the weighted percentile.
Here values define not only the ordering, but the fraction of the total
sum, and percentile is the fraction of sum of cumulative values
in the weighting column (relative to their sum),
for all ranking colums at or lower than the current ranking column.
If the weight column is not specified
(with C<--mode weighted>), it is the same as the ranking column.

=item B<-a> or B<--include-non-numeric>

Compute stats over all records (treat non-numeric records
as zero rather than just ignoring them).

=item B<-S> or B<--pre-sorted>

Assume data is already sorted.
With one -S, we check and confirm this precondition.
When repeated, we skip the check.

=item B<-N NAME> or B<--new-name NAME>

Give the NAME of the new column.
(If no type is specifed, a type will be assigned based on the mode.)

=item B<-f FORMAT> or B<--format FORMAT>

Specify a L<printf(3)>-style format for output statistics.
Defaults to C<%.5g>.

=item B<-T TmpDir>

where to put tmp files.
Also uses environment variable TMPDIR, if -T is 
not specified.
Default is /tmp.

=item B<-e> EmptyValue or B<--empty>

Specify the value any non-numeric rows get, if in weighted mode.

=back


=for comment
begin_standard_fsdb_options

This module also supports the standard fsdb options:

=over 4

=item B<-d>

Enable debugging output.

=item B<-i> or B<--input> InputSource

Read from InputSource, typically a file name, or C<-> for standard input,
or (if in Perl) a IO::Handle, Fsdb::IO or Fsdb::BoundedQueue objects.

=item B<-o> or B<--output> OutputDestination

Write to OutputDestination, typically a file name, or C<-> for standard output,
or (if in Perl) a IO::Handle, Fsdb::IO or Fsdb::BoundedQueue objects.

=item B<--autorun> or B<--noautorun>

By default, programs process automatically,
but Fsdb::Filter objects in Perl do not run until you invoke
the run() method.
The C<--(no)autorun> option controls that behavior within Perl.

=item B<--help>

Show help.

=item B<--man>

Show full manual.

=back

=for comment
end_standard_fsdb_options


=head1 SAMPLE USAGE

=head2 Input:

    #fsdb name id test1
    a 1 80
    b 2 70
    c 3 65
    d 4 90
    e 5 70
    f 6 90

=head2 Command:

    cat DATA/grades.fsdb | dbcolpercentile test1

=head2 Output:

	#fsdb name id test1 percentile
	d	4	90	1
	f	6	90	1
	a	1	80	0.66667
	b	2	70	0.5
	e	5	70	0.5
	c	3	65	0.16667
	#  | dbsort -n test1
	#   | dbcolpercentile test1

=head2 Command 2:

    cat DATA/grades.fsdb | dbcolpercentile --rank test1

=head2 Output 2:

	#fsdb name id test1 rank
	d	4	90	1
	f	6	90	1
	a	1	80	3
	b	2	70	4
	e	5	70	4
	c	3	65	6
	#  | dbsort -n test1
	#   | dbcolpercentile --rank test1


=head1 SEE ALSO

L<Fsdb>.
L<dbcolhisto>.


=cut


# WARNING: This code is derived from dbcolpercentile.pm; that is the master copy.

use Fsdb::Filter::dbcolpercentile;
my $f = new Fsdb::Filter::dbcolpercentile(@ARGV);
$f->setup_run_finish;  # or could just --autorun
exit 0;


=head1 AUTHOR and COPYRIGHT

Copyright (C) 1997-2024 by John Heidemann <johnh@isi.edu>

This program is distributed under terms of the GNU general
public license, version 2.  See the file COPYING
with the distribution for details.

=cut

1;