23.10.2014-17:30:12 :: Something smarter than us is going to emerge


node:	23.10.2014-17:30:12
template:	4
parent:	LINUX pro uplne zacatecniky
owner:	Geo
viewed by:
created:	23.10.2014 - 17:30:12
updated:	23.10.2014 - 17:30:53

cwbe coordinatez:
101
63540
63542
2109677
63692
7738781

ABSOLUT
KYBERIA

permissions
you:	r,
system:	public
net:	yes

ahoj,
asi nie som uplny zaciatocnik, ale mam problem a neviem si poradit.
mame servery z ktorych, ked dam cp aaa.txt /dev/null tak diksy citaju 250 MBps, ale ked dam scp aaa.txt user@127.0.0.1:/dev/null tak to ide tak 20 MBps, neviem kam sa mam kukat, prosim poradte. ide o to ze transefer je shitty. a som si na 99% isty ze je to config toho serveru, nie targetu, kedze presuvam len na lokal...
dik

00000101000635400006354202109677000636920773878107738807

inak sa to sprava zauijmavo, mame tam len 1 Gbps link a hocktory process kopiruje max tak 20 MBps, ale ked je ich viac tak to vytazia na max 120MBps...

00000101000635400006354202109677000636920773878107738791

ssh sifrovanie zatazuje procesor a moze spomalit prenos. riesenim je povolit slabe arcfour (RC4) sifrovanie, ktore je najrychlejsie. alebo ftp.

0000010100063540000635420210967700063692077387810773879107738802

[root@osdmgpbsdnapp1 mp1]# scp aaa.dat root@127.0.0.1:/dev/null
root@127.0.0.1's password:
aaa.dat 44% 1766MB 17.0MB/s 02:11 ETA

--------------------

-rw-r--r-- 1 root root 1827 Sep 9 2013 xorg.conf.new
[root@osdmgpbsdnapp1 ~]# top
top - 22:39:51 up 4 days, 23:24, 18 users, load average: 1.68, 0.82, 0.60
Tasks: 617 total, 4 running, 606 sleeping, 7 stopped, 0 zombie
Cpu(s): 15.6%us, 3.6%sy, 0.0%ni, 80.3%id, 0.0%wa, 0.0%hi, 0.4%si, 0.0%st
Mem: 32883784k total, 12077028k used, 20806756k free, 257280k buffers
Swap: 8224760k total, 9668k used, 8215092k free, 5297380k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2946 root 20 0 59836 5736 2748 R 67.4 0.0 1:19.24 ssh
2947 root 20 0 97200 5476 2984 R 64.5 0.0 1:17.61 sshd
8614 root 0 -20 295m 15m 1612 S 10.2 0.0 200:53.03 perfd
2945 root 20 0 55072 2032 1512 R 4.3 0.0 0:05.87 scp
8520 root 0 -20 35536 7044 1532 S 4.3 0.0 156:01.64 scopeux
2952 root 20 0 55072 2004 1488 S 1.6 0.0 0:02.23 scp
2978 root 20 0 15428 1632 932 R 1.0 0.0 0:01.30 top
9613 root 20 0 14040 3108 1784 S 0.3 0.0 8:32.55 CFSfsckdAgent
9617 root 20 0 14040 3108 1784 S 0.3 0.0 8:32.44 CVMVxconfigdAge
9618 root 20 0 11992 2976 1816 S 0.3 0.0 8:21.92 HostMonitor
9619 root 20 0 11988 2952 1772 S 0.3 0.0 8:22.73 NICAgent
1 root 20 0 19332 1200 912 S 0.0 0.0 0:07.05 init
2 root 20 0 0 0 0 S 0.0 0.0 0:00.31 kthreadd
3 root RT 0 0 0 0 S 0.0 0.0 0:53.05 migration/0
4 root 20 0 0 0 0 S 0.0 0.0 0:01.96 ksoftirqd/0
5 root RT 0 0 0 0 S 0.0 0.0 0:00.00 migration/0
6 root RT 0 0 0 0 S 0.0 0.0 0:00.00 watchdog/0

0000010100063540000635420210967700063692077387810773879107738799

skusil som pocas prenosu dat "top" v druhom okne a nevyzera, ze by sa cpu islo roztrhat, ale mozno sa pozeram mimo

0000010100063540000635420210967700063692077387810773879107738792

indicie ako mozem toto sknotrolovat?

000001010006354000063542021096770006369207738781077387910773879207738805

mozes si pustit ssh -v user@server a uvidis tam nieco ako:

debug1: kex: server->client arcfour hmac-md5 none
debug1: kex: client->server arcfour hmac-md5 none

nastavit sa to da cez .ssh/config (Ciphers pre dany host) alebo v prikaze pridat "-c arcfour"

00000101000635400006354202109677000636920773878107738791077387920773880507738809

akoze
ssh -v root@127.0.0.1
?

0000010100063540000635420210967700063692077387810773879107738792077388050773880907738812

[root@osdmgpbsdnapp1 ~]# ssh -v root@127.0.0.1
OpenSSH_5.3p1, OpenSSL 1.0.0-fips 29 Mar 2010
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug1: Connecting to 127.0.0.1 [127.0.0.1] port 22.
debug1: Connection established.
debug1: permanently_set_uid: 0/0
debug1: identity file /root/.ssh/identity type -1
debug1: identity file /root/.ssh/id_rsa type 1
debug1: identity file /root/.ssh/id_dsa type -1
debug1: Remote protocol version 2.0, remote software version OpenSSH_5.3
debug1: match: OpenSSH_5.3 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_5.3
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-md5 none
debug1: kex: client->server aes128-ctr hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Host '127.0.0.1' is known and matches the RSA host key.
debug1: Found key in /root/.ssh/known_hosts:26
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
debug1: Next authentication method: gssapi-keyex
debug1: No valid Key exchange context
debug1: Next authentication method: gssapi-with-mic
debug1: Unspecified GSS failure. Minor code may provide more information
Credentials cache file '/tmp/krb5cc_0' not found

debug1: Unspecified GSS failure. Minor code may provide more information
Credentials cache file '/tmp/krb5cc_0' not found

debug1: Unspecified GSS failure. Minor code may provide more information

debug1: Unspecified GSS failure. Minor code may provide more information

debug1: Next authentication method: publickey
debug1: Trying private key: /root/.ssh/identity
debug1: Offering public key: /root/.ssh/id_rsa
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
debug1: Trying private key: /root/.ssh/id_dsa
debug1: Next authentication method: password

000001010006354000063542021096770006369207738781077387910773879207738805077388090773881207738815

by default ti pouzil aes128-ctr, ved si skus ten arcfour a uvidis ci sa to zrychli

00000101000635400006354202109677000636920773878107738791077387920773880507738809077388120773881507738818

napises pls, co presne mam linuxu povedat a vyskusat? dakujem
ked prides do jakarty tak mas u mna zmrzlinu :)

0000010100063540000635420210967700063692077387810773879107738792077388050773880907738812077388150773881807738820

skus pridat do scp prikazu -c arcfour alebo si do .ssh/config pridaj:

Host server
User user
Ciphers arcfour

000001010006354000063542021096770006369207738781077387910773879207738805077388090773881207738815077388180773882007738881

dakujem inak za nasmerovanie :)

000001010006354000063542021096770006369207738781077387910773879207738805077388090773881207738815077388180773882007738831

dodatocna otazka... je sifrovanie singlethread proces? to by vysvetlovalo to ze cpu nie je vytazene...

000001010006354000063542021096770006369207738781077387910773879207738805077388090773881207738815077388180773882007738827

btw, znamena to, ze ten stroj ma slabe CPU?

000001010006354000063542021096770006369207738781077387910773879207738805077388090773881207738815077388180773882007738822

well ide to trochu rychlejsie...
[root@osdmgpbsdnapp1 mp1]# scp -c arcfour aaa.dat root@127.0.0.1:/dev/null
root@127.0.0.1's password:
aaa.dat 4% 199MB 34.8MB/s 01:49 ETA
ale chcel by some tak 100 MBps
je este nejake stupidnejsie sifrovanie (ziadne sifrovanie)?

00000101000635400006354202109677000636920773878107738791077387920773880507738809077388120773881507738818077388200773882207738897

este mozes skusit -o 'Compression no'

k otazkam vyssie, sifrovanie je dost narocna operacia, takze to je normalne. sifry su asi dost tazko paralelizovatelne, plus dalsie limity ako ze CBC zo svojej podstaty musi mat vysledok predchadzajuceho bloku kym moze sifrovat dalsi.

btw 34.8 MB je takmer 280 Mbit (neviem ci si naozaj myslel MBbit) a napriklad u mna to na Gbit sieti je bezna max. hodnota prenosu na mechanickom disku (t.j. ani nesifrovane FTP neda viac). to je podla mna bottleneck u teba, ale neviem co mas za cpu

0000010100063540000635420210967700063692077387810773879107738792077388050773880907738812077388150773881807738820077388220773889707738933

U mna MBps = MegaBajt/s, my sa tu hrame na velky datawarehouse takze chcem mat prenosy 100 - 500 MBps, mechanicke disky su bottleneck len kym nemas RAID, a viem ze tie disky dookazu dat tak 250 MBps, takze to s tvojou pomocou mozem obmedzit na cpu. Netusim co su to za servery, ale asi nejake smejdy a-la vmware lowcost.
Kazdopadne si ma nasmeroval spravnym smerom a dlhujem ti dve zmrzliny :)

00000101000635400006354202109677000636920773878107738789

Red Hat Enterprise Linux Server release 6.1 (Santiago)

00000101000635400006354202109677000636920773878107738785

skusal som iptables -L, ala to dava len standardny vystup