shell bypass 403
l
��bc @ s� d Z d d l Z d d l m Z m Z m Z m Z m Z m Z m Z m
Z
m Z m Z m
Z
m Z m Z m Z d d l m Z d d l m Z d d d d d
d d d
d d d d d d d d d d d d g Z Gd � d � Z Gd � d
e � Z e d
e � Gd � d e � Z e d e � Gd � d e � Z e d e � Gd! � d � Z Gd" � d � Z y e Wn e k
r�e Z Yn XGd# � d � Z d S($ u+
csv.py - read/write/investigate CSV files
i N( u Erroru __version__u writeru readeru register_dialectu unregister_dialectu get_dialectu
list_dialectsu field_size_limitu
QUOTE_MINIMALu QUOTE_ALLu QUOTE_NONNUMERICu
QUOTE_NONEu __doc__( u Dialect( u StringIOu
QUOTE_MINIMALu QUOTE_ALLu QUOTE_NONNUMERICu
QUOTE_NONEu Erroru Dialectu __doc__u excelu excel_tabu field_size_limitu readeru writeru register_dialectu get_dialectu
list_dialectsu Snifferu unregister_dialectu __version__u
DictReaderu
DictWriterc B s\ | Ee Z d Z d Z d Z d Z d Z d Z d Z
d Z d Z d Z
d � Z d � Z d S( u� Describe a CSV dialect.
This must be subclassed (see csv.excel). Valid attributes are:
delimiter, quotechar, escapechar, doublequote, skipinitialspace,
lineterminator, quoting.
u c C s) | j t k r d | _ n | j � d S( NT( u __class__u Dialectu Trueu _validu _validate( u self( ( u* /usr/local/python-3.2/lib/python3.2/csv.pyu __init__) s c C sL y t | � Wn7 t k
rG } z t t | � � � WYd d } ~ Xn Xd S( N( u _Dialectu TypeErroru Erroru str( u selfu e( ( u* /usr/local/python-3.2/lib/python3.2/csv.pyu _validate. s NF( u __name__u
__module__u __doc__u _nameu Falseu _validu Noneu delimiteru quotecharu
escapecharu doublequoteu skipinitialspaceu lineterminatoru quotingu __init__u _validate( u
__locals__( ( u* /usr/local/python-3.2/lib/python3.2/csv.pyu Dialect s
c B s8 | Ee Z d Z d Z d Z d Z d Z d Z e
Z d S( u; Describe the usual properties of Excel-generated CSV files.u ,u "u
NTF( u __name__u
__module__u __doc__u delimiteru quotecharu Trueu doublequoteu Falseu skipinitialspaceu lineterminatoru
QUOTE_MINIMALu quoting( u
__locals__( ( u* /usr/local/python-3.2/lib/python3.2/csv.pyu excel5 s
c B s | Ee Z d Z d Z d S( uE Describe the usual properties of Excel-generated TAB-delimited files.u N( u __name__u
__module__u __doc__u delimiter( u
__locals__( ( u* /usr/local/python-3.2/lib/python3.2/csv.pyu excel_tab? s
u excel-tabc B s8 | Ee Z d Z d Z d Z d Z d Z d Z e
Z d S( u: Describe the usual properties of Unix-generated CSV files.u ,u "u
NTF( u __name__u
__module__u __doc__u delimiteru quotecharu Trueu doublequoteu Falseu skipinitialspaceu lineterminatoru QUOTE_ALLu quoting( u
__locals__( ( u* /usr/local/python-3.2/lib/python3.2/csv.pyu unix_dialectD s
u unix_dialectu unixc B sV | Ee Z d d d d d � Z d � Z e d � � Z e j d � � Z d � Z d S( u excelc O sI | | _ | | _ | | _ t | | | | � | _ | | _ d | _ d S( Ni ( u _fieldnamesu restkeyu restvalu readeru dialectu line_num( u selfu fu
fieldnamesu restkeyu restvalu dialectu argsu kwds( ( u* /usr/local/python-3.2/lib/python3.2/csv.pyu __init__P s c C s | S( N( ( u self( ( u* /usr/local/python-3.2/lib/python3.2/csv.pyu __iter__Y s c C sS | j d k r= y t | j � | _ Wq= t k
r9 Yq= Xn | j j | _ | j S( N( u _fieldnamesu Noneu nextu readeru
StopIterationu line_num( u self( ( u* /usr/local/python-3.2/lib/python3.2/csv.pyu
fieldnames\ s
c C s
| | _ d S( N( u _fieldnames( u selfu value( ( u* /usr/local/python-3.2/lib/python3.2/csv.pyu
fieldnamesf s c C s� | j d k r | j n t | j � } | j j | _ x | g k rX t | j � } q: Wt t | j | � � } t | j � } t | � } | | k r� | | d � | | j <n: | | k r� x+ | j | d � D] } | j | | <q� Wn | S( Ni ( u line_numu
fieldnamesu nextu readeru dictu zipu lenu restkeyu restval( u selfu rowu du lfu lru key( ( u* /usr/local/python-3.2/lib/python3.2/csv.pyu __next__j s
N( u __name__u
__module__u Noneu __init__u __iter__u propertyu
fieldnamesu setteru __next__( u
__locals__( ( u* /usr/local/python-3.2/lib/python3.2/csv.pyu
DictReaderO s
c B sD | Ee Z d d d d � Z d � Z d � Z d � Z d � Z d S( u u raiseu excelc O s\ | | _ | | _ | j � d k r7 t d | � � n | | _ t | | | | � | _ d S( Nu raiseu ignoreu- extrasaction (%s) must be 'raise' or 'ignore'( u raiseu ignore( u
fieldnamesu restvalu loweru
ValueErroru extrasactionu writer( u selfu fu
fieldnamesu restvalu extrasactionu dialectu argsu kwds( ( u* /usr/local/python-3.2/lib/python3.2/csv.pyu __init__� s
c C s, t t | j | j � � } | j | � d S( N( u dictu zipu
fieldnamesu writerow( u selfu header( ( u* /usr/local/python-3.2/lib/python3.2/csv.pyu writeheader� s c sd � j d k rJ � f d � � D� } | rJ t d d j | � � � qJ n � � f d � � j D� S( Nu raisec s% g | ] } | � j k r | � q S( ( u
fieldnames( u .0u k( u self( u* /usr/local/python-3.2/lib/python3.2/csv.pyu
<listcomp>� s u( dict contains fields not in fieldnames: u , c s% g | ] } � j | � j � � q S( ( u getu restval( u .0u key( u rowdictu self( u* /usr/local/python-3.2/lib/python3.2/csv.pyu
<listcomp>� s ( u extrasactionu
ValueErroru joinu
fieldnames( u selfu rowdictu wrong_fields( ( u rowdictu selfu* /usr/local/python-3.2/lib/python3.2/csv.pyu
_dict_to_list� s c C s | j j | j | � � S( N( u writeru writerowu
_dict_to_list( u selfu rowdict( ( u* /usr/local/python-3.2/lib/python3.2/csv.pyu writerow� s c C s= g } x$ | D] } | j | j | � � q
W| j j | � S( N( u appendu
_dict_to_listu writeru writerows( u selfu rowdictsu rowsu rowdict( ( u* /usr/local/python-3.2/lib/python3.2/csv.pyu writerows� s
N( u __name__u
__module__u __init__u writeheaderu
_dict_to_listu writerowu writerows( u
__locals__( ( u* /usr/local/python-3.2/lib/python3.2/csv.pyu
DictWriter� s
c B sD | Ee Z d Z d � Z d d � Z d � Z d � Z d � Z d S( ue
"Sniffs" the format of a CSV file (i.e. delimiter, quotechar)
Returns a Dialect object.
c C s d d d d d g | _ d S( Nu ,u u ;u u :( u preferred( u self( ( u* /usr/local/python-3.2/lib/python3.2/csv.pyu __init__� s c C s� | j | | � \ } } } } | s? | j | | � \ } } n | sT t d � � n Gd � d t � } | | _ | | _ | p� d | _ | | _ | S( uI
Returns a dialect (or None) corresponding to the sample
u Could not determine delimiterc B s | Ee Z d Z d Z e Z d S( u sniffedu
N( u __name__u
__module__u _nameu lineterminatoru
QUOTE_MINIMALu quoting( u
__locals__( ( u* /usr/local/python-3.2/lib/python3.2/csv.pyu dialect� s
u dialectu "( u _guess_quote_and_delimiteru _guess_delimiteru Erroru Dialectu doublequoteu delimiteru quotecharu skipinitialspace( u selfu sampleu
delimitersu quotecharu doublequoteu delimiteru skipinitialspaceu dialect( ( u* /usr/local/python-3.2/lib/python3.2/csv.pyu sniff� s c C s& g } xC d D]; } t j | t j t j B� } | j | � } | r
Pq
q
W| sV d Si } i } d } x| D]� } | j d d }
| |
} | r� | j | d � d | | <n y | j d d }
| |
} Wn t k
r� wo Yn X| r%| d k s| | k r%| j | d � d | | <n y | j d
d }
Wn t k
rQwo Yn X| |
ro | d 7} qo qo Wt
| d | j �} | r�t
| d | j �}
| |
| k } |
d k r�d }
q�n d }
d } t j d
i |
d 6| d 6t j � } | j | � rd } n d } | | |
| f S( u�
Looks for text enclosed between two identical quotes
(the probable quotechar) which are preceded and followed
by the same character (the probable delimiter).
For example:
,'some text',
The quote with the most wins, same with the delimiter.
If there is no quotechar the delimiter can't be determined
this way.
uF (?P<delim>[^\w
"'])(?P<space> ?)(?P<quote>["']).*?(?P=quote)(?P=delim)uC (?:^|
)(?P<quote>["']).*?(?P=quote)(?P<delim>[^\w
"'])(?P<space> ?)uD (?P<delim>>[^\w
"'])(?P<space> ?)(?P<quote>["']).*?(?P=quote)(?:$|
)u* (?:^|
)(?P<quote>["']).*?(?P=quote)(?:$|
)u i u quotei u delimu spaceu keyu
u] ((%(delim)s)|^)\W*%(quote)s[^%(delim)s\n]*%(quote)s[^%(delim)s\n]*%(quote)s\W*((%(delim)s)|$)( uF (?P<delim>[^\w
"'])(?P<space> ?)(?P<quote>["']).*?(?P=quote)(?P=delim)uC (?:^|
)(?P<quote>["']).*?(?P=quote)(?P<delim>[^\w
"'])(?P<space> ?)uD (?P<delim>>[^\w
"'])(?P<space> ?)(?P<quote>["']).*?(?P=quote)(?:$|
)u* (?:^|
)(?P<quote>["']).*?(?P=quote)(?:$|
)FN( u FNi T(
u reu compileu DOTALLu MULTILINEu findallu Falseu Noneu
groupindexu getu KeyErroru maxu searchu True( u selfu datau
delimitersu matchesu restru regexpu quotesu delimsu spacesu mu nu keyu quotecharu delimu skipinitialspaceu dq_regexpu doublequote( ( u* /usr/local/python-3.2/lib/python3.2/csv.pyu _guess_quote_and_delimiter� s\
c C s� t t d | j d � � � } d � t d � D� } t d t | � � } d } i } i } i } d t | t | � � } }
x�| t | � k r | d 7} xn | | |
� D]\ } xS | D]K } | j | i � }
| j | � } |
j | d � d |
| <|
| | <q� Wq� Wx� | j � D]� } t | | j
� � } t | � d k rb| d d d k rbqn t | � d k r�t | d d � �| | <| j | | � | | d | | d t
d � | D� � f | | <q| d | | <qW| j
� } t | | � } d
} d } x� t | � d k r�| | k r�xp | D]h \ } } | d d k r7| d d k r7| d | | k r�| d k s�| | k r�| | | <q�q7q7W| d 8} qWt | � d k r
t | j � � d } | d j | � | d j d
| � k } | | f S|
} |
| 7}
q� W| s+d St | � d k r�xZ | j D]L } | | j � k rG| d j | � | d j d
| � k } | | f SqGWn d � | j
� D� } | j � | d d } | d j | � | d j d
| � k } | | f S( u�
The delimiter /should/ occur the same number of times on
each row. However, due to malformed data, it may not. We don't want
an all or nothing approach, so we allow for small variations in this
number.
1) build a table of the frequency of each character on every line.
2) build a table of frequencies of this frequency (meta-frequency?),
e.g. 'x occurred 5 times in 10 rows, 6 times in 1000 rows,
7 times in 2 rows'
3) use the mode of the meta-frequency to determine the /expected/
frequency for that character
4) find out how often the character actually meets that goal
5) the character that best meets its goal is the delimiter
For performance reasons, the data is evaluated in chunks, so it can
try and evaluate the smallest portion of the data possible, evaluating
additional chunks as necessary.
u
c S s g | ] } t | � � q S( ( u chr( u .0u c( ( u* /usr/local/python-3.2/lib/python3.2/csv.pyu
<listcomp>- s i i
i i u keyc S s | d S( Ni ( ( u x( ( u* /usr/local/python-3.2/lib/python3.2/csv.pyu <lambda>G s c s s | ] } | d Vq d S( i N( ( u .0u item( ( u* /usr/local/python-3.2/lib/python3.2/csv.pyu <genexpr>L s g �?g�������?g{�G�z�?u %c u c S s" g | ] \ } } | | f � q S( ( ( u .0u ku v( ( u* /usr/local/python-3.2/lib/python3.2/csv.pyu
<listcomp>v s N( u i i����( u listu filteru Noneu splitu rangeu minu lenu getu countu keysu itemsu maxu removeu sumu floatu preferredu sort( u selfu datau
delimitersu asciiu chunkLengthu iterationu
charFrequencyu modesu delimsu startu endu lineu charu
metaFrequencyu frequ itemsu modeListu totalu consistencyu thresholdu ku vu delimu skipinitialspaceu d( ( u* /usr/local/python-3.2/lib/python3.2/csv.pyu _guess_delimiter st
&! !
c
C s t t | � | j | � � } t | � } t | � } i } x t | � D] } d | | <qI Wd } x� | D]� } | d k r� Pn | d 7} t | � | k r� qj n x� t | j � � D]� } xU t t
t g D]4 }
y |
| | � PWq� t t
f k
r� Yq� Xq� Wt | | � }
|
| | k r� | | d k r@|
| | <qJ| | =q� q� Wqj Wd } x� | j � D]� \ } } t | � t d � k r�t | | � | k r�| d 7} q�| d 8} qey | | | � Wn" t t f k
r�| d 7} YqeX| d 8} qeW| d k S( Ni i i ( u readeru StringIOu sniffu nextu lenu rangeu Noneu listu keysu intu floatu complexu
ValueErroru
OverflowErroru itemsu typeu TypeError(
u selfu sampleu rdru headeru columnsu columnTypesu iu checkedu rowu colu thisTypeu hasHeaderu colType( ( u* /usr/local/python-3.2/lib/python3.2/csv.pyu
has_header sJ
N( u __name__u
__module__u __doc__u __init__u Noneu sniffu _guess_quote_and_delimiteru _guess_delimiteru
has_header( u
__locals__( ( u* /usr/local/python-3.2/lib/python3.2/csv.pyu Sniffer� s
J g( u __doc__u reu _csvu Erroru __version__u writeru readeru register_dialectu unregister_dialectu get_dialectu
list_dialectsu field_size_limitu
QUOTE_MINIMALu QUOTE_ALLu QUOTE_NONNUMERICu
QUOTE_NONEu Dialectu _Dialectu iou StringIOu __all__u excelu excel_tabu unix_dialectu
DictReaderu
DictWriteru complexu NameErroru floatu Sniffer( ( ( u* /usr/local/python-3.2/lib/python3.2/csv.pyu <module> s. ^
2!