python爬蟲(chóng)selenium模塊詳解

更新時(shí)間：2021年03月30日 09:42:04 作者：南岸青梔*

這篇文章主要介紹了python爬蟲(chóng)selenium模塊詳解,本文給大家介紹的非常詳細(xì)，對(duì)大家的學(xué)習(xí)或工作具有一定的參考借鑒價(jià)值，需要的朋友可以參考下

selenium模塊

selenium基本概念

selenium優(yōu)勢(shì)

便捷的獲取網(wǎng)站中動(dòng)態(tài)加載的數(shù)據(jù)
便捷實(shí)現(xiàn)模擬登陸

selenium使用流程：

1.環(huán)境安裝：pip install selenium

2.下載一個(gè)瀏覽器的驅(qū)動(dòng)程序（谷歌瀏覽器）

3.實(shí)例化一個(gè)瀏覽器對(duì)象

基本使用

代碼

from selenium import webdriver
from lxml import etree
from time import sleep

if __name__ == '__main__':

 bro = webdriver.Chrome(r"E:\google\Chrome\Application\chromedriver.exe")
 bro.get(url='http://scxk.nmpa.gov.cn:81/xk/')

 page_text = bro.page_source
 tree = etree.HTML(page_text)
 li_list = tree.xpath('//*[@id="gzlist"]/li')
 for li in li_list:
  name = li.xpath('./dl/@title')[0]
  print(name)
 sleep(5)
 bro.quit()

基于瀏覽器自動(dòng)化的操作

代碼

#編寫(xiě)基于瀏覽器自動(dòng)化的操作代碼

- 發(fā)起請(qǐng)求: get(url)

- 標(biāo)簽定位: find系列的方法

- 標(biāo)簽交互: send_ keys( 'xxx' )

- 執(zhí)行js程序: excute_script('jsCod')

- 前進(jìn)，后退: back(),forward( )

- 關(guān)閉瀏覽器: quit()

代碼

https://www.taobao.com/

from selenium import webdriver
from time import sleep

bro = webdriver.Chrome(executable_path=r"E:\google\Chrome\Application\chromedriver.exe")

bro.get(url='https://www.taobao.com/')

#標(biāo)簽定位
search_input = bro.find_element_by_id('q')
sleep(2)
#執(zhí)行一組js代碼，使得滾輪向下滑動(dòng)
bro.execute_script('window.scrollTo(0,document.body.scrollHeight)')
sleep(2)
#標(biāo)簽交互
search_input.send_keys('女裝')
button = bro.find_element_by_class_name('btn-search')
button.click()

bro.get('https://www.baidu.com')
sleep(2)
bro.back()
sleep(2)
bro.forward()
sleep(5)
bro.quit()

selenium處理iframe：

- 如果定位的標(biāo)簽存在于iframe標(biāo)簽之中，則必須使用switch_to.frame(id)

- 動(dòng)作鏈(拖動(dòng)) : from selenium. webdriver import ActionChains
	- 實(shí)例化一個(gè)動(dòng)作鏈對(duì)象: action = ActionChains (bro)
	- click_and_hold(div) :長(zhǎng)按且點(diǎn)擊操作
	- move_by_offset(x,y)
	- perform( )讓動(dòng)作鏈立即執(zhí)行
	- action.release( )釋放動(dòng)作鏈對(duì)象

代碼

https://www.runoob.com/try/try.php?filename=jqueryui-api-droppable

from selenium import webdriver
from time import sleep
from selenium.webdriver import ActionChains
bro = webdriver.Chrome(executable_path=r"E:\google\Chrome\Application\chromedriver.exe")

bro.get('https://www.runoob.com/try/try.php?filename=jqueryui-api-droppable')

bro.switch_to.frame('iframeResult')

div = bro.find_element_by_id('draggable')

#動(dòng)作鏈
action = ActionChains(bro)
action.click_and_hold(div)

for i in range(5):
 action.move_by_offset(17,0).perform()
 sleep(0.3)

#釋放動(dòng)作鏈
action.release()

bro.quit()

selenium模擬登陸QQ空間

代碼

https://qzone.qq.com/

from selenium import webdriver
from time import sleep


bro = webdriver.Chrome(executable_path=r"E:\google\Chrome\Application\chromedriver.exe")
bro.get('https://qzone.qq.com/')
bro.switch_to.frame("login_frame")

switcher = bro.find_element_by_id('switcher_plogin')
switcher.click()

user_tag = bro.find_element_by_id('u')
password_tag = bro.find_element_by_id('p')
user_tag.send_keys('1234455')
password_tag.send_keys('qwer123')
sleep(1)

but = bro.find_element_by_id('login_button')
but.click()

無(wú)頭瀏覽器和規(guī)避檢測(cè)

代碼

from selenium import webdriver
from time import sleep
#實(shí)現(xiàn)無(wú)可視化界面
from selenium.webdriver.chrome.options import Options
#實(shí)現(xiàn)規(guī)避檢測(cè)
from selenium.webdriver import ChromeOptions

#實(shí)現(xiàn)無(wú)可視化界面
chrome_options = Options()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--disable-gpu')
#實(shí)現(xiàn)規(guī)避檢測(cè)
option = ChromeOptions()
option.add_experimental_option('excludeSwitches',['enable-automation'])

bro = webdriver.Chrome(executable_path=r"E:\google\Chrome\Application\chromedriver.exe",chrome_options=chrome_options,options=option)

bro.get('https://www.baidu.com')
print(bro.page_source)
sleep(2)
bro.quit()

到此這篇關(guān)于python爬蟲(chóng)selenium模塊詳解的文章就介紹到這了,更多相關(guān)python爬蟲(chóng)selenium模塊內(nèi)容請(qǐng)搜索腳本之家以前的文章或繼續(xù)瀏覽下面的相關(guān)文章希望大家以后多多支持腳本之家！

您可能感興趣的文章: