web automation / post to login :: Remember, remember, the velvet November


node:	web automation / post to login
template:	4
parent:	Python Programming Language
owner:	chaplin
viewed by:
created:	06.11.2018 - 07:57:18
updated:	06.11.2018 - 08:00:14

cwbe coordinatez:
101
63540
2076399
856608
8561529

ABSOLUT
KYBERIA

permissions
you:	r,
system:	public
net:	yes

neurons
stats|by_visit|by_K source tiamat K|my_K|given_K last commanders polls
total descendants::8
total children::3
show[ 2 | 3] flat

Chcel by som pomocou requests/beautifulsoup v Python 3 prejst cez nalogovanie do webstranky.

Tu je login/password tlacitko:
..input maxlength="100" size="30" id="loginName" name="loginName" type="text" title="E-mail ID:" value=""
..input maxlength="32" size="30" id="password" name="password" title="Password:" type="password" value=""

Tu je submit button:
..input value="Submit" name="button_okButton" type="submit" onclick="javascript:login(['Mode','Topic'],['login','100'])"

Doleuvedeny kod mi nacita uvodnu stranku, ale neprihlasi ma. Aj ked som zahashoval ten posledny geturl request, tak aj tak to neukaze dalsiu stranku. Viem sa tam dostat a vsetko vytahat cez selenium/chrome, ale chcel by som nieco co by pracovalo na pozadi...

import ssl
import requests
from lxml import html

USERNAME = "myusername@domain.com"
PASSWORD = "password"

LOGIN_URL = "https://ehnsarmciratihs.extnet.ibm.com/services/security/cirats/index.jsp"
URL = "https://ehnsarmciratihs.extnet.ibm.com/services/security/cirats/index.jsp"

def main():
ssl._create_default_https_context = ssl._create_unverified_context
session_requests = requests.session()

# Create payload
payload = {
"loginName": USERNAME,
"password": PASSWORD,
#"csrfmiddlewaretoken": authenticity_token
}

# Perform login
result = session_requests.post(LOGIN_URL, data = payload, headers = dict(referer = LOGIN_URL))

# Scrape url
result = session_requests.get(URL, headers = dict(referer = URL), verify=False)
print(result.text)

if __name__ == '__main__':
main()

000001010006354002076399008566080856152908594055

Neries to v Pythone. https://github.com/GoogleChrome/puppeteer

00000101000635400207639900856608085615290859405508598527

dakujem, skusim

000001010006354002076399008566080856152908563288

ulkas 09.11.2018 - 22:08:55 [1K] , level: 1, UP NEW

Re: web automation / post to login

nevies si proste odchytit samotny vygenerovany request na stranke? a modifikovat uz len request header a nie simulovat samotne prihlasenie cez form submit. tu je napriklad request header pre prihlasenie na kyberiu:





Request URL: https://kyberia.sk/id/8561529/

Request Method: POST

Status Code: 302 

Remote Address: 85.118.132.159:443

Referrer Policy: same-origin

:authority: kyberia.sk

:method: POST

:path: /id/8561529/

:scheme: https

accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8

accept-encoding: gzip, deflate, br

accept-language: en-US,en;q=0.9,sk;q=0.8

cache-control: max-age=0

content-length: 652

content-type: multipart/form-data; boundary=----WebKitFormBoundaryxxx

cookie: PHPSESSID=xxx

origin: https://kyberia.sk

referer: https://kyberia.sk/id/8561529

upgrade-insecure-requests: 1

user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.87 Safari/537.36

------WebKitFormBoundaryxxx

Content-Disposition: form-data; name="login"



ulkas

------WebKitFormBoundaryxxx

Content-Disposition: form-data; name="password"



xxx

------WebKitFormBoundaryxxx

Content-Disposition: form-data; name="event"



login

------WebKitFormBoundaryxxx

Content-Disposition: form-data; name="login_type"



name

------WebKitFormBoundaryxxx

Content-Disposition: form-data; name="screen_width"



1920

------WebKitFormBoundaryxxx

Content-Disposition: form-data; name="screen_height"



1025

------WebKitFormBoundaryxxx--

toto vies postovat aj primitivnym curlom v konzole. odpoved zo servera je nastavena session cookie tusim

000001010006354002076399008566080856152908561557

javascript:login :)

mam k tomu len taku anekdotu, ze asi pred 10 rokmi som uspesne robil amazon login cez ruby-webkit. neviem co myslis tym na pozadi, ale ten proces nemal viditelne okno a bezal propri vsetkych ostatnych procesoch... mysli na to, ze content za tym loginom moze byt (a pravdepodobne bude) generovany javascriptom.

00000101000635400207639900856608085615290856155708561638

dakujem, vies to nejak zakomponovat do toho kodu? Som v tychto veciach hodne novy, jedna vec je urobit v seleniu find_element_by_id_executescript("...") a druha urobit nieco v takomto kode ktory som si pozical z inej stranky a neviem ako zmenit post metodu na volanie javascriptu v tomto kontexte. Skusim to pogooglit...

edit: a este sa potrebujem dostat do JSON a tohoto :D
https://developers.trello.com/docs/api-introduction

0000010100063540020763990085660808561529085615570856163808561643

ja o tom nic neviem, spustal som js kod v tej stranke aby som vyplnil formular a submitol (dalsim js). dnes na to snad existuju nejake frameworky, urcite nie si prvy kto riesi taketo peklo. radsej si double-checkni ci na to co chces ziskat neexistuje nejake API, scraping sa moze lahko rozbit.

000001010006354002076399008566080856152908561557085616380856164308561645

jj prave mi jeden indian potvrdil ze maju API :) dakujem

0000010100063540020763990085660808561529085615570856163808561641

nasiel som javascript:login v ciratsScripts.js

function login(argumentName, argumentValues){
dm1 = document.MainForm;
var dt = new Date();

// Adding the timeZone information to the request..
argumentName[argumentName.length]='timeZone';
argumentValues[argumentValues.length] = dt.getTimezoneOffset();
//Added by Deepmala for defect 38731:Problem with timestamp display for some DST timezones
var jan1 = new Date(dt.getFullYear(), 0, 1, 0, 0, 0, 0);
var temp = jan1.toGMTString();
var jan2 = new Date(temp.substring(0, temp.lastIndexOf(" ")-1));
var time_offset_jan = (jan1 - jan2) / (1000 * 60);
var june1 = new Date(dt.getFullYear(), 6, 1, 0, 0, 0, 0);
temp = june1.toGMTString();
var june2 = new Date(temp.substring(0, temp.lastIndexOf(" ")-1));
var time_offset_jun = (june1 - june2) / (1000 * 60);

argumentName[argumentName.length]='time_offset_jan';
argumentValues[argumentValues.length] = time_offset_jan;
argumentName[argumentName.length]='time_offset_jun';
argumentValues[argumentValues.length] = time_offset_jun;

//End of code added by Deepmala for defect 38731:Problem with timestamp display for some DST timezones
submitParameters(argumentName,argumentValues);
}