IZjJTUddlmZddlZddlZddlZddlmZddlm Z ddl m Z ddl m Z mZmZmZmZmZddlmZmZdd lmZmZdd lmZmZmZmZdd lmZdd l m!Z!dd l"m#Z#dZ$dZ%dZ&dZ'ia(e e)e#fe*d<ej+dZ,de e)e#ffdZ-de)de#ddfdZ.de)ddfdZ/e dd d8de0de)de0fd Z1d9de2fd!Z3 d:de0d"e2d#eede e)e#ffd$Z4Gd%d&eZ5Gd'd(e5Z6Gd)d*e5Z7dddde6fd+ee)efde0d,e0d-e0d.e0d#eedeeee)e2efeddefffd/Z8 d;d+ee)efde0d,e0d-e0d.e0d#eedee)fd0Z9 d)unicode_literalsN)open) lru_cache)isabs)DictListOptionalTupleTypeUnion) SplitResulturlsplit)BaseTLDSourceParserRegistry) TldBadUrlTldDomainNotFoundTldImproperlyConfigured TldIOError) project_dir)Result)TriezArtur Barseghyanz2013-2026 Artur Barseghyanz,MPL-1.1 OR GPL-2.0-only OR LGPL-2.1-or-later)BaseMozillaTLDSourceParser MozillaPublicOnlyTLDSourceParserMozillaTLDSourceParserrget_fldget_tld get_tld_namesget_tld_names_containeris_tld parse_tldpop_tld_names_container process_url protocol_rereset_tld_names tld_namesupdate_tld_namesupdate_tld_names_cliupdate_tld_names_containerr&z^[a-z0-9.+:-]*//returnctS)zCGet container of all tld names. :return: :rtype dict: r&U/Users/jameslopez/projects/MentorCore/.venv/lib/python3.11/site-packages/tld/utils.pyrr6s  r.tld_names_local_pathtrie_objc|t|<dS)ziUpdate TLD Names container item. :param tld_names_local_path: :param trie_obj: :return: Nr,)r0r1s r/r)r)@s'/I"###r.c<t|ddS)zTRemove TLD names container item. :param tld_names_local_path: :return: N)r&popr0s r/r"r"NsMM&-----r.T)maxsizetypedF fail_silently parser_uidc4g}|j}|r>tj|d}|r&|jr|||nAtjD]-\}}|r&|jr|||.t |S)zUUpdate TLD names. :param fail_silently: :param parser_uid: :return: Nr9)appendrget source_urlr'itemsall)r9r:resultsresults_append parser_clss r/r'r'XsG^N \*d33  */  N++-+HH   '/n&6&6   "J  j3 //m/LL w<<r.cBtjd}|dddd|dd d d d |||ntjdd}|j}|j}tt|| S)zCLI wrapper for update_tld_names. Since update_tld_names returns True on success, we need to negate the result to match CLI semantics. zUpdate TLD names) descriptionr:?Nz*UID of the parser to update TLD names for.)nargsdefaulthelpz--fail-silentlyr9F store_truez Fail silently)destrIactionrJr)r:r9) argparseArgumentParser add_argument parse_argssysargvr:r9intr')rSparserargsr:r9s r/r(r(ts  $1C D D DF  9         T%5TT38ABB< H HDJ&M   - P P PP  r. retry_count parser_classcB|st}|||S)aBuild the ``tlds`` list if empty. Recursive. :param fail_silently: If set to True, no exceptions are raised and None is returned on failure. :param retry_count: If greater than 1, we raise an exception in order to avoid infinite loops. :param parser_class: :type fail_silently: bool :type retry_count: int :type parser_class: BaseTLDSourceParser :return: List of TLD names :rtype: obj:`tld.utils.Trie` r9rW)rr)r9rWrXs r/rrs3$ .-  % %# &  r.c ReZdZe ddededeeee ffdZ dS) rFrr9rWr*c|dkr |rdStt}|j|vr||j|S t|jr|j}nt |j}t |dd}t }|j}d}|j} |D]W} d| vrd}|r| snJd | vr| d} | d d vr7|| | Xt|j|| n#t$rZ|| |dz }|||cY | S#t $rYSwxYwt $r5} |r,Yd} ~ | dS#t $rYdSwxYw| d} ~ wwxYw | n:#t $rYn.wxYw# | w#t $rYwwxYwxYw|S)z[Parse. :param fail_silently: :param retry_count: :return: rNrutf8)encodingFz===BEGIN PRIVATE DOMAINS===Tz// xn--r)/ )privater<rZ)rr& local_pathrr codecs_openraddinclude_privatesplitstripr)closeIOErrorr'r Exception) clsr9rW _tld_namesrc local_filetrietrie_addprivate_sectionrflineerrs r/rz(BaseMozillaTLDSourceParser.get_tld_namess ?? !t   Nj ( (3>*6 2 S^$$ 9 ^ (88 $ZvFFFJ66DxH#O!1O" E E0D88&*O"?E$$::<<?D7k))DJJLL*ODDDDD &s~t < < <            } = = = 1 K$$+%     """"        ttt   """""            """"       """"    sC#DG!NN N7:N $sDy/ "NNN[NNNr.rc>eZdZUdZdZeed<dZeed<dZeed<dS) rMozilla TLD source.mozillauidz4https://publicsuffix.org/list/public_suffix_list.datr?zres/effective_tld_names.dat.txtrcN) rtrurv__doc__r}ry__annotations__r?rcr-r.r/rrsHCLJLLL7J77777r.rcLeZdZUdZdZeed<dZeed<dZeed<dZ e ed <d S) rr{mozilla_public_onlyr}z?https://publicsuffix.org/list/public_suffix_list.dat?publiconlyr?z+res/effective_tld_names_public_only.dat.txtrcFrfN) rtrurvr~r}ryrr?rcrfrxr-r.r/rr sd$C$$$IDJCCC!OT!!!!!r.rurl fix_protocol search_publicsearch_privatec |s|stdt||}t|tsd|r1t|sd|} t|}n"#t$r}|r dd|fcYd}~S|d}~wwxYw|}|j } | s|rdd|fSt|| } | dr| d} | d} |j} || j} d} d}d}t!| }t#|dz d d D]l}| |}| jnZ|| jkrnM| j|}|| jd }|n| dz } |} | jr| }| }m||jr|s|jr |s|jr|rdd|fSt/| ||krd }nt1d||z }| ||fS) zProcess URL. :param parser_class: :param url: :param fail_silently: :param fix_protocol: :param search_public: :param search_private: :return: zJEither `search_public` or `search_private` (or both) shall be set to True.)r9rXzhttps://N)r.rr*) domain_name)rr isinstancer r$matchlowerr ValueErrorhostnamerendswithrstriprgrcrootlenrangechildren exceptionr>leafrbrmax)rr9rrrrXrm parsed_urler domain_partsr0nodecurrent_length tld_lengthrlen_domain_partsipartchild non_zero_is r/r#r#s$  ^ %    #,J c; ' '   # 1 1#))++ > > #"S""C !#JJ    T3&&&&&&    %K %  %z) )$$$ $##%%KC  .!((-- $$S))L'2 * + 0DNJ E<(( #a'R 0 0A = E 4> ! ! E !!$'' =M%%c**E = E ! 9 'JE    &+m  $)=   =z) )# <<< <:%% ,z9:: Z //s$.A>> BBBBBc d|vrtd|s|rtnt}t||||||\}}} |dS|J|dkr| jSd||dz dS)a?Extract the first level domain. Extract the top level domain based on the mozilla's effective TLD names dat file. Returns a string. May throw ``TldBadUrl`` or ``TldDomainNotFound`` exceptions if there's bad URL provided or no TLD match found respectively. :param url: URL to get top level domain from. :param fail_silently: If set to True, no exceptions are raised and None is returned on failure. :param fix_protocol: If set to True, missing or wrong protocol is ignored (https is appended instead). :param search_public: If set to True, search in public domains. :param search_private: If set to True, search in private domains. :param parser_class: :type url: str | SplitResult :type fail_silently: bool :type fix_protocol: bool :type search_public: bool :type search_private: bool :return: String with top level domain (if ``as_object`` argument is set to False) or a ``tld.utils.Result`` object (if ``as_object`` argument is set to True); returns None on failure. :rtype: str as_objectzH`as_object` argument is deprecated for `get_fld`. Use `get_tld` instead.rr9rrrrXNrrr)rrrr#rjoin) rr9rrrrXkwargsrrrs r/rrsDf%      2 " "1  ,7 #!#%! ,,,(L*jt  ! ! !A~~"" 88La!1!12 3 33r.rc|s|rtnt}t||||||\}}} |dS|J|s*|dkr| jSd||dS|dkrd} d} | j Jd| j} n]d|d|dz } d||dz |} d||d} t | | | | S) aExtract the top level domain. Extract the top level domain based on the mozilla's effective TLD names dat file. Returns a string. May throw ``TldBadUrl`` or ``TldDomainNotFound`` exceptions if there's bad URL provided or no TLD match found respectively. :param url: URL to get top level domain from. :param fail_silently: If set to True, no exceptions are raised and None is returned on failure. :param as_object: If set to True, ``tld.utils.Result`` object is returned, ``domain``, ``suffix`` and ``tld`` properties. :param fix_protocol: If set to True, missing or wrong protocol is ignored (https is appended instead). :param search_public: If set to True, search in public domains. :param search_private: If set to True, search in private domains. :param parser_class: :type url: str | SplitResult :type fail_silently: bool :type as_object: bool :type fix_protocol: bool :type search_public: bool :type search_private: bool :return: String with top level domain (if ``as_object`` argument is set to False) or a ``tld.utils.Result`` object (if ``as_object`` argument is set to True); returns None on failure. :rtype: str rNrrzNo hostname in URLr) subdomaindomaintldr)rrr#rrr) rr9rrrrrXrrrrr_tlds r/rrsTJ   2 " "1  ,7 #!#%! ,,,(L*jt  ! ! ! 3 >>& &xx Z[[1222A~~ "..0D..."HH\*:JN*:;<< ,zA~ 'BCDDxx Z[[122 F   r.NNNc |s|rtnt} t||d||||}|dS|j|j|jfS#t tttf$rYnwxYwdS)zParse TLD into parts. :param url: :param fail_silently: :param fix_protocol: :param search_public: :param search_private: :param parser_class: :return: Tuple (tld, domain, subdomain) :rtype: tuple T)r9rrrrrXNr) rrrrrrrrrr)rr9rrrrXobjs r/r!r!)s&   2 " "1    '%')%    ;##w CM11 (*A: N       sAAA"!A"valuec\|s|rtnt}t|dd|||}||kS)agCheck if given URL is tld. :param value: URL to get top level domain from. :param search_public: If set to True, search in public domains. :param search_private: If set to True, search in private domains. :param parser_class: :type value: str :type search_public: bool :type search_private: bool :return: :rtype: bool Tr)rrr)rrrrXrs r/r r XsX$   2 " "1   #%!    D D=r.c0|rt|dSiadS)zReset the ``tld_names`` to empty value. If ``tld_names_local_path`` is given, removes specified entry from ``tld_names`` instead. :param tld_names_local_path: :type tld_names_local_path: str :return: N)r"r&r5s r/r%r%|s, 455555 r.)FN)N)FrN)FFTTN)FFFTTN)TTN)> __future__rrNrerRcodecsrrd functoolsros.pathrtypingrrr r r r urllib.parser rbaserr exceptionsrrrrhelpersrresultrror __author__ __copyright__ __license____all__r&ryrcompiler$rr)r"rxr'rTr(rrrrr#rrr!r r%r-r.r/rsQ''''''' &&&&&&;;;;;;;;;;;;;;;;........//////// !   , <  * 4T ?bj,-- c4i / /)- /  / / / /.#.$.... 3d###37-0 $#6s: .2*+ #t)_ >PPPPP!4PPPf888887888"""""'A"""$ .D s0s0 sK s0s0s0 s0  s0 *+ s0 5cC, -uT45L/M MNs0s0s0s0p .2 B4B4 sK B4B4B4 B4  B4 *+ B4c]B4B4B4B4N .2QQ sK QQQ Q  Q  Q*+QeCK !QQQQl .2 ,, sK ,,, ,  , *+ , 5! "E#sC-$8 89,,,,b.2 !! k! "!!!*+ !  !!!!H#r.