(ph 6 % S r SrSSKJr SSKJr SSKrSSKJr SSKrSSK J
r
J r SSK J
r
SS KJrJrJrJrJrJrJrJrJrJr SS
KJr SS KJ r J r SSK r Sr \\
\ S ' SSK!r!\!r S
\%S\\& 4S jr'Sr(\&\ S' Sr)\&\ S' \*" 5 r+\\\\&\4 4 \ S' \RX " \)R[ S5 \R\ 5 \RX " \(R[ S5 \R\ 5 S.\+\%' \RX " \)\R\ 5 \RX " \(\R\ 5 S.\+\&' " S S\/5 r0\0Rc 5 " S S5 r2 " S S 5 r3g! \" a/ SSK#r#\#r GN
! \" a SSK$r$\$r GN! \" a GN&f = ff = ff = f) a~ Beautiful Soup bonus library: Unicode, Dammit
This library converts a bytestream to Unicode through any means
necessary. It is heavily based on code from Mark Pilgrim's `Universal
Feed Parser `_, now maintained
by Kurt McKee. It does not rewrite the body of an XML or HTML document
to reflect a new encoding; that's the job of `TreeBuilder`.
MIT )codepoint2name) defaultdictN)html5)Logger getLogger)
ModuleType)
DictIteratorListOptionalPatternSetTupleTypeUnioncast)Literal) _Encoding
_Encodingschardet_modulesreturnc p [ b [ U [ 5 ( a g[ nUR U 5 S $ )z?Try as hard as possible to detect the encoding of a bytestring.Nencoding)r
isinstancestrdetect)r modules =/var/www/html/venv/lib/python3.13/site-packages/bs4/dammit.py_chardet_dammitr! G s0 As!3!3
F == J '' z$^\s*<\?.*encoding=['"](.*?)['"].*\?> xml_encodingz0<\s*meta[^>]+charset\s*=\s*["']?([^>]*?)[ /;'">] html_meta encoding_resascii)htmlxmlc \ rS rSr% Sr\\\4 \S' \\\4 \S' \\ \S' \\ \S' \ S%S j5 r
S
S S S
SS.r \\\4 \S' \ R " S\ R 5 r\ R " S5 r\\ \S' \ R " S5 r\\ \S' \ S\ R$ S\4S j5 r\ S\ R$ S\4S j5 r\ S\ R$ S\4S j5 r\ S\ R$ S\4S j5 r\ S\S\4S j5 r\ S&S\S \S\4S jj5 r\ S&S\S \S\4S jj5 r\ S \S\4S! j5 r\ S \S\4S" j5 r \ S \S\4S# j5 r S$r g)'EntitySubstitutionb zFThe ability to substitute XML or HTML entities for certain characters.HTML_ENTITY_TO_CHARACTERCHARACTER_TO_HTML_ENTITYCHARACTER_TO_HTML_ENTITY_RE*CHARACTER_TO_HTML_ENTITY_WITH_AMPERSAND_REr Nc 0 n0 n[ 5 n[ [ 5 n[ [ R " 5 5 H u pVUR S5 ( a USS nOUnXr; a XbU' XqU' [
U5 S:X a [ U5 S: a US; a MV [
U5 S: a [ S U 5 5 ( a M~ [
U5 S:X a US:w a UR U5 M XFS R U5 M [ 5 nU H^ n XI n
U
( d UR U 5 M! S
R U
V s/ s H oS PM sn 5 n UR U < S U < S 35 M` [ UR 5 5 H n
U
H nUR U5 M M S
SR U5 - nUR S5 S
SR U5 - n[ [ R " 5 5 H u nn[ U5 nXqU' M Xl X l [" R$ " U5 U l [" R$ " U5 U l gs sn f )ue Initialize variables used by this class to manage the plethora of
HTML5 named entities.
This function sets the following class variables:
CHARACTER_TO_HTML_ENTITY - A mapping of Unicode strings like "⦨" to
entity names like "angmsdaa". When a single Unicode string has
multiple entity names, we try to choose the most commonly-used
name.
HTML_ENTITY_TO_CHARACTER: A mapping of entity names like "angmsdaa" to
Unicode strings like "⦨".
CHARACTER_TO_HTML_ENTITY_RE: A regular expression matching (almost) any
Unicode string that corresponds to an HTML5 named entity.
CHARACTER_TO_HTML_ENTITY_WITH_AMPERSAND_RE: A very similar
regular expression to CHARACTER_TO_HTML_ENTITY_RE, but which
also matches unescaped ampersands. This is used by the 'html'
formatted to provide backwards-compatibility, even though the HTML5
spec allows most ampersands to go unescaped.
;N z<>c 3 > # U H n[ U5 S : v M g7f)r4 N)ord).0xs r ?EntitySubstitution._populate_class_variables.. s )J 1#a&3, s &r z(?![z])z(%s)|)setr sortedr itemsendswithlenr6 alladdjoinlistvaluesr chrr- r, recompiler. r/ )clsunicode_to_namename_to_unicodeshort_entities long_entities_by_first_charactername_with_semicolon charactername particlesshort
long_versionsr8 ignore
long_entities long_entity
re_definition re_definition_with_ampersand codepoints r _populate_class_variables,EntitySubstitution._populate_class_variables s# 0 +6s+;(.4U[[].C * #++C00*3B/*
*(1%
*.I & 9~"s9~'; QU@U
9~!c)J )J&J&J 9~"yC'7 ""9-01 >BB9M{ /DB E #E < CM
e$
!>
1A$
!>?
eV <= $ ""B"I"I"K LM ,
k* - M )!44
c'-0C'C$ $N$8$8$: ;OItII)-I & < (7$'6$*,**]*C'9; (:
67 "?s 8I
aposquotampltgt)'"r; <>CHARACTER_TO_XML_ENTITYz&(#\d+|#x[0-9a-fA-F]+|\w+);z&([<>]|&(?!#\d+;|#x[0-9a-fA-F]+;|\w+;))BARE_AMPERSAND_OR_BRACKETz([<>&])AMPERSAND_OR_BRACKETmatchobjc t UR S5 nU R R U5 nUc SU- $ SU- $ )zhUsed with a regular expression to substitute the
appropriate HTML entity for a special character string.r &%s;&%s;)groupr- get)rK rj original_entityentitys r _substitute_html_entity*EntitySubstitution._substitute_html_entity sB #..+--11/B > / / r" c H U R UR S5 nSU- $ )zgUsed with a regular expression to substitute the
appropriate XML entity for a special character string.r rm )rg rn )rK rj rq s r _substitute_xml_entity)EntitySubstitution._substitute_xml_entity) s' ,,X^^A->? r" c * SUR S5 - $ )Nrl r3 )rn )rK rj s r _escape_entity_name&EntitySubstitution._escape_entity_name0 s HNN1 ---r" c V UR S5 nX R ; a SU- $ SU- $ )Nr3 rm rl )rn r, )rK rj possible_entitys r _escape_unrecognized_entity_name3EntitySubstitution._escape_unrecognized_entity_name4 s2 "..+ : : :O+ +O++r" valuec Z SnSU; a SU; a SnUR SU5 nOSnX!- U- $ )a# Make a value into a quoted XML attribute, possibly escaping it.
Most strings will be quoted using double quotes.
Bob's Bar -> "Bob's Bar"
If a string contains double quotes, it will be quoted using
single quotes.
Welcome to "my bar" -> 'Welcome to "my bar"'
If a string contains both single and double quotes, the
double quotes will be escaped, and the string will be quoted
using double quotes.
Welcome to "Bob's Bar" -> Welcome to "Bob's bar"
:param value: The XML attribute value to quote
:return: The quoted value
rd rc z")replace)rK r~
quote_with replace_withs r quoted_attribute_value)EntitySubstitution.quoted_attribute_value; sC ,
%<e| (
c<8 !
!J..r" make_quoted_attributec U R R U R U5 nU( a U R U5 nU$ )a Replace special XML characters with named XML entities.
The less-than sign will become <, the greater-than sign
will become >, and any ampersands will become &. If you
want ampersands that seem to be part of an entity definition
to be left alone, use `substitute_xml_containing_entities`
instead.
:param value: A string to be substituted.
:param make_quoted_attribute: If True, then the string will be
quoted, as befits an attribute value.
:return: A version of ``value`` with special characters replaced
with named entities.
)ri subru r rK r~ r s r substitute_xml!EntitySubstitution.substitute_xmlc s: & ((,,S-G-GO ..u5E r" c U R R U R U5 nU( a U R U5 nU$ )ag Substitute XML entities for special XML characters.
:param value: A string to be substituted. The less-than sign will
become <, the greater-than sign will become >, and any
ampersands that are not part of an entity defition will
become &.
:param make_quoted_attribute: If True, then the string will be
quoted, as befits an attribute value.
)rh r ru r r s r "substitute_xml_containing_entities5EntitySubstitution.substitute_xml_containing_entities| s: --11#2L2LeT ..u5E r" r c N U R R U R U5 $ )a' Replace certain Unicode characters with named HTML entities.
This differs from ``data.encode(encoding, 'xmlcharrefreplace')``
in that the goal is to make the result more readable (to those
with ASCII displays) rather than to recover from
errors. There's absolutely nothing wrong with a UTF-8 string
containg a LATIN SMALL LETTER E WITH ACUTE, but replacing that
character with "é" will make it more readable to some
people.
:param s: The string to be modified.
:return: The string with some Unicode characters replaced with
HTML entities.
)r/ r rr rK r s r substitute_html"EntitySubstitution.substitute_html s( " ==AA ' '
r" c U R R U R U5 nU R R U R U5 nU$ )a Replace certain Unicode characters with named HTML entities
using HTML5 rules.
Specifically, this method is much less aggressive about
escaping ampersands than substitute_html. Only ambiguous
ampersands are escaped, per the HTML5 standard:
"An ambiguous ampersand is a U+0026 AMPERSAND character (&)
that is followed by one or more ASCII alphanumerics, followed
by a U+003B SEMICOLON character (;), where these characters do
not match any of the names given in the named character
references section."
Unlike substitute_html5_raw, this method assumes HTML entities
were converted to Unicode characters on the way in, as
Beautiful Soup does. By the time Beautiful Soup does its work,
the only ambiguous ampersands that need to be escaped are the
ones that were escaped in the original markup when mentioning
HTML entities.
:param s: The string to be modified.
:return: The string with some Unicode characters replaced with
HTML entities.
)
ANY_ENTITY_REr rx r. rr r s r substitute_html5#EntitySubstitution.substitute_html5 sG 6
! !#"9"91 =
+ + / /0K0KQ Or" c U R R U R U5 nU R R U R U5 nU$ )ae Replace certain Unicode characters with named HTML entities
using HTML5 rules.
substitute_html5_raw is similar to substitute_html5 but it is
designed for standalone use (whereas substitute_html5 is
designed for use with Beautiful Soup).
:param s: The string to be modified.
:return: The string with some Unicode characters replaced with
HTML entities.
)r r r| r. rr r s r substitute_html5_raw'EntitySubstitution.substitute_html5_raw sG
! !#"F"F J
+ + / /0K0KQ Or" )r N)F) __name__
__module__ __qualname____firstlineno____doc__r
r __annotations__r classmethodr\ rg rI rJ Ir rh ri Matchrr ru rx r| r boolr r r r r __static_attributes__r r" r r* r* b sC P
#38n, #38n, ") - 18 <A
A
N
/T#s(^ JJ >EM /1jj8/ ws| *,I)>'#,> rxx C bhh 3 .288 . . . , ,S , , %/3 %/3 %/ %/N 3 t PS 0 7<04 *
( B S S r" r* c r \ rS rSr% Sr SS\S\\ S\\ S\\ S\\ S \\ 4 S
jjr \\
S' \\
S' \\
S' \\ \
S ' \\
S' \\ \
S ' \\
S' \\ \
S
' S\\ S\ \ S\4S jr
\S\\ 4S j5 r\S\S\\\\ 4 4S j5 r\ SS\\\4 S\S\S\\ 4S jj5 rSrg)EncodingDetectori af This class is capable of guessing a number of possible encodings
for a bytestring.
Order of precedence:
1. Encodings you specifically tell EncodingDetector to try first
(the ``known_definite_encodings`` argument to the constructor).
2. An encoding determined by sniffing the document's byte-order mark.
3. Encodings you specifically tell EncodingDetector to try if
byte-order mark sniffing fails (the ``user_encodings`` argument to the
constructor).
4. An encoding declared within the bytestring itself, either in an
XML declaration (if the bytestring is to be interpreted as an XML
document), or in a tag (if the bytestring is to be
interpreted as an HTML document.)
5. An encoding detected through textual analysis by chardet,
cchardet, or a similar external library.
6. UTF-8.
7. Windows-1252.
:param markup: Some markup in an unknown encoding.
:param known_definite_encodings: When determining the encoding
of ``markup``, these encodings will be tried first, in
order. In HTML terms, this corresponds to the "known
definite encoding" step defined in `section 13.2.3.1 of the HTML standard `_.
:param user_encodings: These encodings will be tried after the
``known_definite_encodings`` have been tried and failed, and
after an attempt to sniff the encoding by looking at a
byte order mark has failed. In HTML terms, this
corresponds to the step "user has explicitly instructed
the user agent to override the document's character
encoding", defined in `section 13.2.3.2 of the HTML standard `_.
:param override_encodings: A **deprecated** alias for
``known_definite_encodings``. Any encodings here will be tried
immediately after the encodings in
``known_definite_encodings``.
:param is_html: If True, this markup is considered to be
HTML. Otherwise it's assumed to be XML.
:param exclude_encodings: These encodings will not be tried,
even if they otherwise would be.
Nmarkupknown_definite_encodingsis_htmlexclude_encodingsuser_encodingsoverride_encodingsc [ U=( d / 5 U l U( a/ [ R " S[ SS9 U =R U-
sl U=( d / U l U=( d / n[
U Vs/ s H owR 5 PM sn5 U l S U l Uc SOUU l
S U l U R U5 u U l
U l g s snf )NzcThe 'override_encodings' argument was deprecated in 4.10.0. Use 'known_definite_encodings' instead. )
stacklevelF)rF r warningswarnDeprecationWarningr r> lowerr chardet_encodingr declared_encodingstrip_byte_order_markr sniffed_encoding)selfr r r r r r r8 s r __init__EncodingDetector.__init__ s )--E-K(L% MMu"
) )-? ? ) , 2 - 3!$9J%K9JAggi9J%K!L $ 'uW 04 .2-G-G-O* T*
&Ls 4Cr r r r triedr c z Uc gUR 5 nXR ; a gX; a UR U5 gg)zShould we even bother to try this encoding?
:param encoding: Name of an encoding.
:param tried: Encodings that have already been tried. This
will be modified as a side effect.
FT)r r rD )r r r s r _usableEncodingDetector._usable@ s@ >># -- - IIh r" c # L # [ 5 nU R H nU R X!5 ( d M Uv M! U R b/ U R U R U5 ( a U R v U R H nU R X!5 ( d M Uv M! U R
c+ U R
U R U R 5 U l U R
b/ U R U R
U5 ( a U R
v U R c [ U R 5 U l U R b/ U R U R U5 ( a U R v S H nU R X!5 ( d M Uv M! g7f)zYield a number of encodings that might work for this markup.
:yield: A sequence of strings. Each is the name of an encoding
that *might* work to convert a bytestring into Unicode.
N)utf-8 windows-1252) r> r r r r r find_declared_encodingr r r r! )r r es r encodingsEncodingDetector.encodingsQ sj !$ ..A||A%% / , ! !52
2
'' ' $$A||A%% % ! ! )%)%@%@ T\\&D " ! ! -$,, " "E3
3
(( ( ($3DKK$@D ! , ! !52
2
'' ' +A||A%% +s .F$A'F$ C8F$ F$datac T Sn[ U[ 5 ( a X4$ [ U5 S: a USS S:X a USS S:w a
SnUSS nX4$ [ U5 S: a USS S:X a USS S:w a
SnUSS nX4$ USS S
:X a
S nUS S nX4$ USS S :X a
S
nUSS nX4$ USS S:X a SnUSS nX4$ )zIf a byte-order mark is present, strip it and return the encoding it implies.
:param data: A bytestring that may or may not begin with a
byte-order mark.
:return: A 2-tuple (data stripped of byte-order mark, encoding implied by byte-order mark)
N s s zutf-16bes zutf-16ler s r s zutf-32bes zutf-32le)r r rB )rK r r s r r &EncodingDetector.strip_byte_order_mark s dC > !
Y!^bq[(ak)!H8D" ~ Y!^bq[(ak)!H8D ~ "1X
( H8D ~
"1X,
,!H8D ~ "1X,
,!H8D~ r" search_entire_documentc U( a
[ U5 =pEO#Sn[ S[ [ U5 S- 5 5 n[ U[ 5 ( a [
[ nO
[
[ nUS nUS nSn UR XS9n
U
( d U( a UR XS9n
U
b U
R 5 S n U ( a7 [ U [ 5 ( a U R S S
5 n U R 5 $ g) a Given a document, tries to find an encoding declared within the
text of the document itself.
An XML encoding is declared at the beginning of the document.
An HTML encoding is declared in a tag, hopefully near the
beginning of the document.
:param markup: Some markup.
:param is_html: If True, this markup is considered to be HTML. Otherwise
it's assumed to be XML.
:param search_entire_document: Since an encoding is supposed
to declared near the beginning of the document, most of
the time it's only necessary to search a few kilobytes of
data. Set this to True to force this method to search the
entire document.
:return: The declared encoding, if one is found.
i i g?r( r' N)endposr r& r ) rB maxintr bytesr% r searchgroupsdecoder ) rK r r r
xml_endpos html_endposresxml_rehtml_rer declared_encoding_matchs r r 'EncodingDetector.find_declared_encoding s 2 "'*6{ 2J J dCF d(:$; > @ C +U33$5$<$"'26/337PP #+:"6P $ P
$J/ P !,
P %Z 0P6 )( !! y))
M ** M y)) 3 C N t " 18I. 1 1f # # 5@S9S3T # # J ', /eSj!/ / !% /
) / /r" r c \ rS rSr% Sr/ SS/ SS4S\S\\ S\\S S \ S
\\ S \\ S \\ 4S
jjr
\\ S' \\ \ S' \ \ S' \\
\ S' \\ \ S' \\\
\ 4 \ S' \\ S' S\R$ S\4S jrSSS.r\\ \
4 \ S' / SQr\\ S' GSS \
S \ S\\ 4S jjr GSS \S \
S \ S\ 4S! jjr\S\\
4S" j5 rS#\
S\\ 4S$ jrS#\
S\\ 4S% jr 0 S&S'_S(S)_S*S+_S,S-_S.S/_S0S1_S2S3_S4S5_S6S7_S8S9_S:S;_SS?_S@SA_SBSC_SDSA_SESA_SFSGSHSISJSKSLSMSNSOSPSQSASRSSST.Er \\\ \ \\ \ 4 4 4 \ SU' 0 S&SV_S(S)_S*SW_S,SX_S.SY_S0SZ_S2S[_S4S\_S6S]_S8S^_S:S__SSa_S@SA_SBSb_SDSA_SESA_0 ScSd_SeSd_SfSg_ShSg_SiSj_SkSl_SmSn_SoSp_SqSr_SsSt_SuSv_SwSx_SySA_SzS{_S|S}_S~S)_SS_E0 SS_SS_SS_SS_SS_SS__SS_SS_SS_SS_SS_SS)_SS_SSl_SS_SS_SS_E0 SS_SSd_SS_SS_SSj_SSW_SS_SS_SS_SS_SS_SS_SSA_SS_SS_SS_SS_E0 SS_SS_SS_SS_SS_SS_SS_SS_SS_SS_SS_SS_SS_SS_SS_SS_SS_E0 SS_SS_SSj_SS_SS_SS_SS_SS_SS}_SS_SS_SS_SS_SS_SS_SS_SS_E0 SS_SS_SS_SS_SS_SS_SS_SS_SS_SS_SS_SS_SS_SS_SS_SS_SS_ESSSSSSSSSS. Er \\\ 4 \ S' 0 SGS _GSGS_GSGS_GSGS_GSGS_GS GS
_GS GS _GS
GS_GSGS_GSGS_GSGS_GSGS_GSGS_GSGS_GSGS _GS GS _GS GS _0 GS!GS"_GS#GS$_GS%GS&_GS'GS(_GS)GS*_GS+GS,_GS-GS._GS/GS0_GS1GS2_GS3GS4_GS5GS6_GS7GS8_GS9GS:_GS;GS<_GS=GS>_GS?GS@_GSAGSB_E0 GSCGSD_GSEGSF_GSGGSH_GSIGSJ_GSKGSL_GSMGSN_GSOGSP_GSQGSR_GSSGST_GSUGSV_GSWGSX_GSYGSZ_GS[GS\_GS]GS^_GS_GS`_GSaGSb_GScGSd_E0 GSeGSf_GSgGSh_GSiGSj_GSkGSl_GSmGSn_GSoGSp_GSqGSr_GSsGSt_GSuGSv_GSwGSx_GSyGSz_GS{GS|_GS}GS~_GSGS_GSGS_GSGS_GSGS_E0 GSGS_GSGS_GSGS_GSGS_GSGS_GSGS_GSGS_GSGS_GSGS_GSGS_GSGS_GSGS_GSGS_GSGS_GSGS_GSGS_GSGS_E0 GSGS_GSGS_GSGS_GSGS_GSGS_GSGS_GSGS_GSS_GSGS_GSGS_GSGS_GSGS_GSGS_GSGS_GSGS_GSGS_GSGS_E0 GSGS_GSGS_GSGS_GSGS_GSGS_GSGS_GSGS_GSGS_GSGS_GSGS_GSGS_GSGS_GSGS_GSGS_GSGS_GSGS_GSGS_EGSGSGSGS.Er \\!\4 \ GS' / GSQr"\\\!\!\!4 \ GS' \"GS GS r#\!\ GS' \"GS GS r$\!\ GS' \% GSGS\GS\
GS\
S\4GS jj5 r&GSr'g(
UnicodeDammiti a A class for detecting the encoding of a bytestring containing an
HTML or XML document, and decoding it to Unicode. If the source
encoding is windows-1252, `UnicodeDammit` can also replace
Microsoft smart quotes with their HTML or XML equivalents.
:param markup: HTML or XML markup in an unknown encoding.
:param known_definite_encodings: When determining the encoding
of ``markup``, these encodings will be tried first, in
order. In HTML terms, this corresponds to the "known
definite encoding" step defined in `section 13.2.3.1 of the HTML standard `_.
:param user_encodings: These encodings will be tried after the
``known_definite_encodings`` have been tried and failed, and
after an attempt to sniff the encoding by looking at a
byte order mark has failed. In HTML terms, this
corresponds to the step "user has explicitly instructed
the user agent to override the document's character
encoding", defined in `section 13.2.3.2 of the HTML standard `_.
:param override_encodings: A **deprecated** alias for
``known_definite_encodings``. Any encodings here will be tried
immediately after the encodings in
``known_definite_encodings``.
:param smart_quotes_to: By default, Microsoft smart quotes will,
like all other characters, be converted to Unicode
characters. Setting this to ``ascii`` will convert them to ASCII
quotes instead. Setting it to ``xml`` will convert them to XML
entity references, and setting it to ``html`` will convert them
to HTML entity references.
:param is_html: If True, ``markup`` is treated as an HTML
document. Otherwise it's treated as an XML document.
:param exclude_encodings: These encodings will not be considered,
even if the sniffing code thinks they might make sense.
NFr r smart_quotes_to)r& r( r' r r r r c X0l / U l SU l X@l [ [
5 U l [ UUUUUU5 U l [ U[ 5 ( d US:X a Xl [ U5 U l S U l
g U R R U l S nU R R H/ n U R R nU R U 5 nUc M/ O U( d\ U R R HB n U S:w a U R U S5 nUc M U R R! S5 SU l O Uc S U l
S U l g Xl g )NFr" r& r zSSome characters could not be decoded, and were replaced with REPLACEMENT CHARACTER.T)r tried_encodings contains_replacement_charactersr r r logr detectorr r r unicode_markuporiginal_encodingr
_convert_fromwarning)
r r r r r r r r ur s
r r UnicodeDammit.__init__ sI / ! /4, X&( $
fc " "fm K"%f+D %)D " mm**
//H]]))F""8,A} 0 !MM 3 3w& **8Y?A=HH$$?
<@D8 4(
9%)D ""&D "# r" r r r r r matchr c UR S5 nU R S:X a3 X R ; a U R U R 5 nU$ Un U$ X R ; a U R U n[ U5 [ L aG U R S:X a SUS R 5 - S- nU$ SUS R 5 - S- n U$ [ [ U5 nUR 5 n U$ UnU$ )zChanges a MS smart quote character to an XML or HTML
entity, or an ASCII character.
TODO: Since this is only used to convert smart quotes, it
could be simplified, and MS_CHARS_TO_ASCII made much less
parochial.
r3 r& r( s &#x ; &r ) rn r MS_CHARS_TO_ASCIIencodeMS_CHARStypetupler r )r r origr
substitutionss r _sub_ms_charUnicodeDammit._sub_ms_charl s kk!n 7 *---,,T299;(
!
}}$ $
d 3
&%/++u4 $}Q'7'>'>'@ @4 G
#]1%5%<%<%> > E
%)m$> z1/4 z1/2 z3/4 A AE C E r D N O U b B a ae r i n /y) r r4 s € s ‚ s ƒ s „ s … s † s ‡ s ˆ s ‰ s Š s ‹ s Œ s Ž s ‘ s ’ s “ s ” s • s – s — s ˜ s ™ s š s › s œ s ž s Ÿ s s ¡ s ¢ s £ s ¤ s ¥ s ¦ s § s ¨ s © s ª s « s ¬ s s ® s ¯ s ° s ± s ² s ³ s ´ s µ s ¶ s · s ¸ s ¹ s º s » s ¼ s ½ s ¾ s ¿ s À s Á s  s à s Ä s Å s Æ s Ç s È s É s Ê s Ë s Ì s Í s Î s Ï s Ð s Ñ s Ò s Ó s Ô s Õ s Ö s × s Ø s Ù s Ú s Û s Ü s Ý s Þ s ß s à s â s ã s ä s å s æ s ç s è s é s ê s ë s ì s í s î s ï s ð s ñ s ò s ó s ô s õ s ö s ÷ s ø s ù s ú s ûs üs ýs þ) WINDOWS_1252_TO_UTF8))r0 rM r )rN r] r )r^ rb r MULTIBYTE_MARKERS_AND_SIZESr FIRST_MULTIBYTE_MARKERr2 r3 LAST_MULTIBYTE_MARKERin_bytes
main_encodingembedded_encodingc ~ UR SS5 R 5 S; a [ S5 eUR 5 S; a [ S5 e/ nSnSnU[ U5 : a X nXpR : a8 XpR
:: a) U R H u pn
Xx: d M
Xy:: d M Xj-
n OV OSUS: aH XpR ; a9 UR XU 5 UR U R U 5 US -
nUnOUS -
nU[ U5 : a M US:X a U$ UR XS
5 S R U5 $ ) a Fix characters from one encoding embedded in some other encoding.
Currently the only situation supported is Windows-1252 (or its
subset ISO-8859-1), embedded in UTF-8.
:param in_bytes: A bytestring that you suspect contains
characters from multiple encodings. Note that this *must*
be a bytestring. If you've already converted the document
to Unicode, you're too late.
:param main_encoding: The primary encoding of ``in_bytes``.
:param embedded_encoding: The encoding that was used to embed characters
in the main document.
:return: A bytestring similar to ``in_bytes``, in which
``embedded_encoding`` characters have been converted to
their ``main_encoding`` equivalents.
r
r )r windows_1252zPWindows-1252 and ISO-8859-1 are the only currently supported embedded encodings.)utf8r z4UTF-8 is the only currently supported main encoding.r r4 r3 Nr" )
r r NotImplementedErrorrB ro rp rn rm r rE ) rK rq rr rs byte_chunks chunk_startposbytestartendsizes r detwingleUnicodeDammit.detwingle; sc . $ $S# . 4 4 6 ?
&&
(9 9%F
CM!=D111d>W>W6W ),(G(G$E} )H $*B*B"B ""8#<= ""3#;#;D#ABq ! q+ CM!, ! O
x 5 6xx $$r" ) r r r r r r r r r )strict)rv r )(r r r r r r r
r r r r r r r r r r rI r r r r
r r r r r r r r r r rm r rn ro rp r r r r r" r r r sd &V : ?B+@ AB+B CB+D EB+F GB+H IB+J KB+N OB+P QB+R SB+T UB+V WB+X YB+Z [B+\ ]B+^ _B+` aB+b cB+d eB+f gB+h iB+j kB+l mB+n oB+p qB+r sB+t uB+v wB+x yB+z {B+| }B+~ B+@ AB+B CB+D EB+F GB+H IB+J KB+L MB+N OB+P QB+R SB+T UB+V WB+X YB+Z [B+\ ]B+^ _B+` aB+b cB+d eB+f gB+h iB+j kB+l mB+n oB+p qB+r sB+t uB+v wB+x yB+z {B+| }B+~ B+@ AB+B CB+D EB+F GB+H IB+J KB+L MB+N OB+P QB+R SB+T UB+V WB+X YB+Z [B+\ ]B+^ _B+` aB+b cB+d eB+f gB+h iB+j kB+l mB+n oB+p qB+r CB+tE3J' BV{. o{. o{.
k{.
o {.
o {.
o
{.
o{.
k{.
o{.
k{.
o{.
k{.
k{.
o {.
o {.
o!{."
o#{.$
o%{.&
o'{.(
o){.*
k+{.,
o-{..
k/{.0
o1{.2
k3{.4
k5{.6
k7{.8
k9{.:
k;{.<
k={.>
k?{.@
kA{.B
kC{.D
kE{.F
kG{.H
kI{.J
kK{.L
kM{.N
kO{.P
kQ{.R
kS{.T
kU{.V
kW{.X
kY{.Z
k[{.\
k]{.^
k_{.`
ka{.b
kc{.d
ke{.f
kg{.h
ki{.j
kk{.l
km{.n
ko{.p
kq{.r
ks{.t
ku{.v
kw{.x
ky{.z
k{{.|
k}{.~
k{.@
kA{.B
kC{.D
kE{.F
kG{.H
kI{.J
kK{.L
kM{.N
kO{.P
kQ{.R
kS{.T
kU{.V
kW{.X
kY{.Z
k[{.\
k]{.^
k_{.`
ka{.b
kc{.d
ke{.f
kg{.h
ki{.j
kk{.l
km{.n
ko{.p
kq{.r
ks{.t
ku{.v
kw{.x
ky{.z
g{{.|
k}{.~
k{.@
kA{.B
kC{.D
kE{.F
kG{.H
kI{.J
kK{.L
kM{.N
kO{.P
kQ{.R
kS{.T
kU{.V
kW{.X
kY{.Z
k[{.\
k]{.^
k_{.`
ka{.b
kc{.d
ke{.f
kg{.h
ki{.j
kk{.l
km{.n
ko{.p u{.$sEz* {|? eCcM&:!; #>a"@"CCC "=R!@!C3C $*'5 D%D% !D% % D%
D% D%r" r )4r __license__
html.entitiesr collectionsr r r rI loggingr r typesr typingr
r r r
r r r r r r typing_extensionsr bs4._typingr r r r r cchardet ImportErrorchardetcharset_normalizerr r r! r# r$ dictr% rJ r r objectr* r\ r r r r" r r s ( #
% & (,$ + N&(u (# ( @ c ?;
3
04v d4c7l++, 5 JJy''0"$$ 7
::l))'2BDD 9 U
JJy"$$ '
::lBDD ) S } }@ , , .v vrb
% b
%S % /N
sH #E# #F*E33F:F FF
F FFFF