NiceLeeのBlog 用爱发电 bilibili~

Python 关于requests的DNS和SNI设置

2021-02-27
nIceLee

阅读:


在不考虑HTTP2等协议需求的情况下,requests包算是个人小项目中比较常用了,怎么能比较方便指定DNS和设置是否发送SNI呢?

前言

  • HTTP请求就算了,直接改header里的host即可。
  • 其实disable发送SNI也不是很实用,毕竟基本上都是套了CDN的,
    而CDN上绑定N多域名是一件很常见的事情。
    对于SNI缺失的请求处理极少是返回默认证书,大多直接返回error。
    但也有例外。
  • 对于指定DNS,因为requests依赖于urllib3,故而网上hook urllib的解决方案是行得通的。
  • 对于SNI,有个解决方案是将SSL的相关参数置为false,可能有些版本可行,
    但有的版本实现似乎有所改变,需要再做更改。
      # 网上最多的解决方案
      import ssl
      ssl.HAS_SNI
    

示例代码

'''
自定义DNS
'''
from urllib3.util import connection
host_table = {
    #"www.baidu.com": "127.0.0.1",
    "e-hentai.org": "104.20.135.21",
}
_orig_create_connection = connection.create_connection 
def _dns_resolver(host):
    if host in host_table:
        print("自定义DNS 解析被调用")
        return host_table[host]
    else:
        return host
def patched_create_connection(address, *args, **kwargs): 
    host, port = address 
    hostname = _dns_resolver(host) 
    return _orig_create_connection((hostname, port), *args, **kwargs) 

def injectDNS(hosts: dict = host_table):
    host_table.update(hosts)
    connection.create_connection = patched_create_connection
def recoverDNS():
    connection.create_connection = _orig_create_connection
    host_table.clear()

'''
取消SNI发送
'''
import urllib3
urllib3.disable_warnings(urllib3.exceptions.SNIMissingWarning)
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
def unsendSNI():
    urllib3.contrib.pyopenssl.HAS_SNI = False
    urllib3.contrib.pyopenssl.inject_into_urllib3()

def sendSNI():
    urllib3.contrib.pyopenssl.HAS_SNI = True
    urllib3.contrib.pyopenssl.extract_from_urllib3()

'''
取消证书 hostname 验证
'''
import urllib3
_origin_match_hostname = urllib3.connection._match_hostname
def _match_hostname(cert, asserted_hostname):
    #print("_do_nothing")
    pass
def uncheck_hostname():
    urllib3.connection._match_hostname = _match_hostname
def check_hostname():
    urllib3.connection._match_hostname = _origin_match_hostname

if __name__ == '__main__':
    import requests
    headers = {
        "Connection": "keep-alive",
		"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",
		"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:82.0) Gecko/20100101 Firefox/82.0",
		"Host": "e-hentai.org",
		"Accept-Encoding": "gzip, deflate",
		"Accept-Language": "zh-CN,en-US;q=0.8",
    }
    injectDNS()
    unsendSNI()
    #uncheck_hostname()
    content = requests.get("https://e-hentai.org").text
    print(content)
    
    recoverDNS()
    sendSNI()
    #check_hostname()

内容
隐藏