dorian
/
paramiko
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

[project @ Arch-1:robey@lag.net--2003-public%secsh--dev--1.0--patch-156] 

rewrite channel pipes to work on windows
the pipe system i was using for simulating an os-level FD (for select) was
retarded.  i realized this week that i could just use a single byte in the
pipe to signal "data is ready" and not try to feed all incoming data thru
the pipe -- and then i don't have to try to make the pipe non-blocking (which
should make it work on windows).  a lot of duplicate code got removed and now
it's all going thru the same code-path on read.

there's still a slight penalty on incoming feeds and calling 'recv' when a
pipe has been opened (by calling 'fileno'), but it's tiny.

removed a bunch of documentation and comments about things not working on
windows, since i think they probably do now.
This commit is contained in:
Robey Pointer 2005-03-26 05:53:00 +00:00
parent 3e5bd84cc5
commit 5d8d1938fa
4 changed files with 34 additions and 255 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
MANIFEST.in
View File

@ -1,3 +1,3 @@
include ChangeLog LICENSE test.py demo.py demo_simple.py demo_server.py demo_windows.py forward.py include ChangeLog LICENSE test.py demo.py demo_simple.py demo_server.py forward.py
recursive-include docs * recursive-include docs *
recursive-include tests *.py *.key recursive-include tests *.py *.key

27
README
View File

@ -47,17 +47,10 @@ also think it will work on Windows, though i've never tested it there. if
you run into Windows problems, send me a patch: portability is important you run into Windows problems, send me a patch: portability is important
to me. to me.
the Channel object supports a "fileno()" call so that it can be passed
into select or poll, for polling on posix. once you call "fileno()" on a
Channel, it changes behavior in some fundamental ways, and these ways
require posix. so don't call "fileno()" on a Channel on Windows. this is
detailed in the documentation for the "fileno" method.
python 2.2 may work, thanks to some patches from Roger Binns. things to python 2.2 may work, thanks to some patches from Roger Binns. things to
watch out for: watch out for:
* sockets in 2.2 don't support timeouts, so the 'select' module is * sockets in 2.2 don't support timeouts, so the 'select' module is
imported to do polling. this may not work on windows. (works fine on imported to do polling.
osx.)
* logging is mostly stubbed out. it works just enough to let paramiko * logging is mostly stubbed out. it works just enough to let paramiko
create log files for debugging, if you want them. to get real logging, create log files for debugging, if you want them. to get real logging,
you can backport python 2.3's logging package. Roger has done that you can backport python 2.3's logging package. Roger has done that
@ -102,32 +95,23 @@ connection is not secure!)
the following example scripts get progressively more detailed: the following example scripts get progressively more detailed:
demo_windows.py
executes 'ls' on any remote server, loading the host key from your
openssh key file. (this script works on windows because it avoids
using terminal i/o or the 'select' module.) it also creates a logfile
'demo_windows.log'.
demo_simple.py demo_simple.py
calls invoke_shell() and emulates a terminal/tty through which you can calls invoke_shell() and emulates a terminal/tty through which you can
execute commands interactively on a remote server. think of it as a execute commands interactively on a remote server. think of it as a
poor man's ssh command-line client. (works only on posix [unix or poor man's ssh command-line client.
macosx].)
demo.py demo.py
same as demo_simple.py, but allows you to authenticiate using a same as demo_simple.py, but allows you to authenticiate using a
private key, and uses the long form of some of the API calls. (posix private key, and uses the long form of some of the API calls.
only.)
forward.py forward.py
command-line script to set up port-forwarding across an ssh transport. command-line script to set up port-forwarding across an ssh transport.
(requires python 2.3 and posix.) (requires python 2.3.)
demo_server.py demo_server.py
an ssh server that listens on port 2200 and accepts a login for an ssh server that listens on port 2200 and accepts a login for
'robey' (password 'foo'), and pretends to be a BBS. meant to be a 'robey' (password 'foo'), and pretends to be a BBS. meant to be a
very simple demo of writing an ssh server. (should work on all very simple demo of writing an ssh server.
platforms.)
*** USE *** USE
@ -235,3 +219,4 @@ v0.9 FEAROW
* ctr forms of ciphers are missing (blowfish-ctr, aes128-ctr, aes256-ctr) * ctr forms of ciphers are missing (blowfish-ctr, aes128-ctr, aes256-ctr)
* server mode needs better documentation * server mode needs better documentation
* why are big files so slow to transfer? profiling needed...

129
demo_windows.py
View File

@ -1,129 +0,0 @@
#!/usr/bin/python
# Copyright (C) 2003-2005 Robey Pointer <robey@lag.net>
#
# This file is part of paramiko.
#
# Paramiko is free software; you can redistribute it and/or modify it under the
# terms of the GNU Lesser General Public License as published by the Free
# Software Foundation; either version 2.1 of the License, or (at your option)
# any later version.
#
# Paramiko is distrubuted in the hope that it will be useful, but WITHOUT ANY
# WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
# A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more
# details.
#
# You should have received a copy of the GNU Lesser General Public License
# along with Paramiko; if not, write to the Free Software Foundation, Inc.,
# 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA.
# This demo is like demo_simple.py, but it doesn't try to use select()
# to poll the ssh channel for reading, so it can be used on Windows.
# It logs into a shell, executes "ls", prints out the results, and
# exits.
import sys, os, base64, getpass, socket, traceback
import paramiko
if os.environ.has_key('HOME'):
# unix
HOME = os.environ['HOME']
else:
# windows
HOME = os.environ['HOMEDRIVE'] + os.environ['HOMEPATH']
##### utility functions
def load_host_keys():
filename = HOME + '/.ssh/known_hosts'
keys = {}
try:
f = open(filename, 'r')
except Exception, e:
print '*** Unable to open host keys file (%s)' % filename
return
for line in f:
keylist = line.split(' ')
if len(keylist) != 3:
continue
hostlist, keytype, key = keylist
hosts = hostlist.split(',')
for host in hosts:
if not keys.has_key(host):
keys[host] = {}
if keytype == 'ssh-rsa':
keys[host][keytype] = paramiko.RSAKey(data=base64.decodestring(key))
elif keytype == 'ssh-dss':
keys[host][keytype] = paramiko.DSSKey(data=base64.decodestring(key))
f.close()
return keys
# setup logging
paramiko.util.log_to_file('demo_windows.log')
# get hostname
username = ''
if len(sys.argv) > 1:
hostname = sys.argv[1]
if hostname.find('@') >= 0:
username, hostname = hostname.split('@')
else:
hostname = raw_input('Hostname: ')
if len(hostname) == 0:
print '*** Hostname required.'
sys.exit(1)
port = 22
if hostname.find(':') >= 0:
hostname, portstr = hostname.split(':')
port = int(portstr)
# get username
if username == '':
default_username = getpass.getuser()
username = raw_input('Username [%s]: ' % default_username)
if len(username) == 0:
username = default_username
password = getpass.getpass('Password for %s@%s: ' % (username, hostname))
# get host key, if we know one
hostkeytype = None
hostkey = None
hkeys = load_host_keys()
if hkeys.has_key(hostname):
hostkeytype = hkeys[hostname].keys()[0]
hostkey = hkeys[hostname][hostkeytype]
print 'Using host key of type %s' % hostkeytype
# now, connect and use paramiko Transport to negotiate SSH2 across the connection
try:
t = paramiko.Transport((hostname, port))
t.connect(username=username, password=password, hostkey=hostkey)
chan = t.open_session()
print '*** Here we go!'
print
print '>>> ls'
chan.exec_command('ls')
f = chan.makefile('r+')
for line in f:
print line.strip('\n')
chan.close()
t.close()
except Exception, e:
print '*** Caught exception: ' + str(e.__class__) + ': ' + str(e)
traceback.print_exc()
try:
t.close()
except:
pass
sys.exit(1)

131
paramiko/channel.py
View File

@ -31,12 +31,6 @@ from ssh_exception import SSHException
from file import BufferedFile from file import BufferedFile
# this is ugly, and won't work on windows
def _set_nonblocking(fd):
import fcntl
fcntl.fcntl(fd, fcntl.F_SETFL, os.O_NONBLOCK)
class Channel (object): class Channel (object):
""" """
A secure tunnel across an SSH L{Transport}. A Channel is meant to behave A secure tunnel across an SSH L{Transport}. A Channel is meant to behave
@ -63,7 +57,7 @@ class Channel (object):
subclass of L{Channel}. subclass of L{Channel}.
@param chanid: the ID of this channel, as passed by an existing @param chanid: the ID of this channel, as passed by an existing
L{Transport}. L{Transport}.
@type chanid: int @type chanid: int
""" """
self.chanid = chanid self.chanid = chanid
@ -84,6 +78,7 @@ class Channel (object):
self.name = str(chanid) self.name = str(chanid)
self.logger = util.get_logger('paramiko.chan.' + str(chanid)) self.logger = util.get_logger('paramiko.chan.' + str(chanid))
self.pipe_rfd = self.pipe_wfd = None self.pipe_rfd = self.pipe_wfd = None
self.pipe_set = False
self.event = threading.Event() self.event = threading.Event()
self.combine_stderr = False self.combine_stderr = False
self.exit_status = -1 self.exit_status = -1
@ -504,9 +499,6 @@ class Channel (object):
out = '' out = ''
self.lock.acquire() self.lock.acquire()
try: try:
if self.pipe_rfd != None:
# use the pipe
return self._read_pipe(nbytes)
if len(self.in_buffer) == 0: if len(self.in_buffer) == 0:
if self.closed or self.eof_received: if self.closed or self.eof_received:
return out return out
@ -526,6 +518,9 @@ class Channel (object):
if len(self.in_buffer) <= nbytes: if len(self.in_buffer) <= nbytes:
out = self.in_buffer out = self.in_buffer
self.in_buffer = '' self.in_buffer = ''
if self.pipe_rfd != None:
# clear the pipe, since no more data is buffered
self._clear_pipe()
else: else:
out = self.in_buffer[:nbytes] out = self.in_buffer[:nbytes]
self.in_buffer = self.in_buffer[nbytes:] self.in_buffer = self.in_buffer[nbytes:]
@ -754,24 +749,21 @@ class Channel (object):
def fileno(self): def fileno(self):
""" """
Returns an OS-level file descriptor which can be used for polling and Returns an OS-level file descriptor which can be used for polling, but
reading (but I{not} for writing). This is primaily to allow python's but I{not} for reading or writing). This is primaily to allow python's
C{select} module to work. C{select} module to work.
The first time C{fileno} is called on a channel, a pipe is created to The first time C{fileno} is called on a channel, a pipe is created to
simulate real OS-level file descriptor (FD) behavior. Because of this, simulate real OS-level file descriptor (FD) behavior. Because of this,
two actual FDs are created -- this may be inefficient if you plan to two OS-level FDs are created, which will use up FDs faster than normal.
use many channels. You won't notice this effect unless you open hundreds or thousands of
channels simultaneously, but it's still notable.
@return: a small integer file descriptor @return: an OS-level file descriptor
@rtype: int @rtype: int
@warning: This method causes several aspects of the channel to change @warning: This method causes channel reads to be slightly less
behavior. It is always more efficient to avoid using this method. efficient.
@bug: This does not work on Windows. The problem is that pipes are
used to simulate an open FD, but I haven't figured out how to make
pipes enter non-blocking mode on Windows yet.
""" """
self.lock.acquire() self.lock.acquire()
try: try:
@ -779,12 +771,8 @@ class Channel (object):
return self.pipe_rfd return self.pipe_rfd
# create the pipe and feed in any existing data # create the pipe and feed in any existing data
self.pipe_rfd, self.pipe_wfd = os.pipe() self.pipe_rfd, self.pipe_wfd = os.pipe()
_set_nonblocking(self.pipe_wfd)
_set_nonblocking(self.pipe_rfd)
if len(self.in_buffer) > 0: if len(self.in_buffer) > 0:
x = self.in_buffer self._set_pipe()
self.in_buffer = ''
self._feed_pipe(x)
return self.pipe_rfd return self.pipe_rfd
finally: finally:
self.lock.release() self.lock.release()
@ -876,10 +864,9 @@ class Channel (object):
if self.ultra_debug: if self.ultra_debug:
self._log(DEBUG, 'fed %d bytes' % len(s)) self._log(DEBUG, 'fed %d bytes' % len(s))
if self.pipe_wfd != None: if self.pipe_wfd != None:
self._feed_pipe(s) self._set_pipe()
else: self.in_buffer += s
self.in_buffer += s self.in_buffer_cv.notifyAll()
self.in_buffer_cv.notifyAll()
finally: finally:
self.lock.release() self.lock.release()
@ -1025,83 +1012,19 @@ class Channel (object):
self._log(DEBUG, 'EOF sent') self._log(DEBUG, 'EOF sent')
return return
def _feed_pipe(self, data): def _set_pipe(self):
"you are already holding the lock" "you are already holding the lock"
if len(self.in_buffer) > 0: if self.pipe_set:
self.in_buffer += data return
return self.pipe_set = True
try: os.write(self.pipe_wfd, '*')
n = os.write(self.pipe_wfd, data)
if n < len(data):
# at least on linux, this will never happen, as the writes are
# considered atomic... but just in case.
self.in_buffer = data[n:]
self._check_add_window(n)
self.in_buffer_cv.notifyAll()
return
except OSError, e:
pass
if len(data) > 1:
# try writing just one byte then
x = data[0]
data = data[1:]
try:
os.write(self.pipe_wfd, x)
self.in_buffer = data
self._check_add_window(1)
self.in_buffer_cv.notifyAll()
return
except OSError, e:
data = x + data
# pipe is very full
self.in_buffer = data
self.in_buffer_cv.notifyAll()
def _read_pipe(self, nbytes): def _clear_pipe(self):
"you are already holding the lock" "you are already holding the lock"
try: if not self.pipe_set:
x = os.read(self.pipe_rfd, nbytes) return
if len(x) > 0: os.read(self.pipe_rfd, 1)
self._push_pipe(len(x)) self.pipe_set = False
return x
except OSError, e:
pass
# nothing in the pipe
if self.closed or self.eof_received:
return ''
# should we block?
if self.timeout == 0.0:
raise socket.timeout()
# loop here in case we get woken up but a different thread has grabbed everything in the buffer
timeout = self.timeout
while not self.closed and not self.eof_received:
then = time.time()
self.in_buffer_cv.wait(timeout)
if timeout != None:
timeout -= time.time() - then
if timeout <= 0.0:
raise socket.timeout()
try:
x = os.read(self.pipe_rfd, nbytes)
if len(x) > 0:
self._push_pipe(len(x))
return x
except OSError, e:
pass
pass
def _push_pipe(self, nbytes):
# successfully read N bytes from the pipe, now re-feed the pipe if necessary
# (assumption: the pipe can hold as many bytes as were read out)
if len(self.in_buffer) == 0:
return
if len(self.in_buffer) <= nbytes:
os.write(self.pipe_wfd, self.in_buffer)
self.in_buffer = ''
return
x = self.in_buffer[:nbytes]
self.in_buffer = self.in_buffer[nbytes:]
os.write(self.pipe_wfd, x)
def _unlink(self): def _unlink(self):
if self.closed or not self.active: if self.closed or not self.active: