split sftp into sftp, sftp_client; renamed SFTP -> SFTPClient
add sftp_client file, and split out the common code (sftp) from stuff specific
to client mode (sftp_client). renamed SFTP class to SFTPClient, but left an
alias so old code will still work.
renamed a bunch of sftp constants now that they're better hidden from epydoc.
some framework for adding subsystem handlers in server mode
you can now register a subsystem with a Transport by passing in the name
(like "sftp") and a class (like a hypothetical SFTPServer). the default
ServerInterface.check_channel_request_subsystem now checks this table in
Transport, and if it finds a match, it creates a new thread for the handler
and calls into it. a new class SubsystemHandler is added for this purpose
(to be subclassed).
remove redundant 'auth_complete' member
remove the redundant 'auth_complete' field and just use 'authenticated' for
both client and server mode. this makes the repr() string look correct in
server mode instead of always claiming that the transport is un-auth'd.
clean up server interface; no longer need to subclass Channel
- export AUTH_*, OPEN_FAILED_*, and the new OPEN_SUCCEEDED into the paramiko
namespace instead of making people dig into paramiko.Transport.AUTH_* etc.
- move all of the check_* methods from Channel to ServerInterface so apps
don't need to subclass Channel anymore just to run an ssh server
- ServerInterface.check_channel_request() returns an error code now, not a
new Channel object
- fix demo_server.py to follow all these changes
- fix a bunch of places where i used "string" in docstrings but meant "str"
- added Channel.get_id()
clean up SecurityOptions
the preferences are now tuples in Transport, and passed as tuples out of
SecurityOptions, so that the options can't be modified without setting them
back to the options field again. the algorithm lists in Transport are used
to validate the fields.
added Transport.get_security_options()
just something i wanted to play with:
added Transport.get_security_options() which returns a SecurityOptions object.
this object is a kind of proxy for the 4 "preferred_*" fields in Transport,
and lets me avoid exposing those fields directly in case i change my mind
later about how they should be stored.
added some docs to Channel explaining that the request methods now return
True/False, and fixed up docs in a few other places.
replay patch 63 (missing channel changes)
i'm still getting the hang of tla/arch, obviously.
replay patch 63, which was meant to be part of the later mega-patch, but
apparently when i reversed it, i lost it entirely.
new ServerInterface class, outbound rekey works, etc.
a bunch of changes that i'm too lazy to split out into individual patches:
* all the server overrides from transport.py have been moved into a separate
class ServerInterface, so server code doesn't have to subclass the whole
paramiko library
* updated demo_server to subclass ServerInterface
* when re-keying during a session, block other messages until the new keys
are activated (openssh doensn't like any other traffic during a rekey)
* re-key when outbound limits are tripped too (was only counting inbound
traffic)
* don't log scary things on EOF
add settimeout/gettimeout/setblocking, some bugfixes.
hide the command and response codes in sftp so they aren't exported.
add settimeout/gettimeout/setblocking that just wrap calls to the underlying
socket or channel. fix _read_all to not catch timeout exceptions.
add settimeout/gettimeout/setblocking, some bugfixes.
hide the command and response codes in sftp so they aren't exported.
add settimeout/gettimeout/setblocking that just wrap calls to the underlying
socket or channel. fix _read_all to not catch timeout exceptions.
limit read/write requests to 32KB, advertise 32KB max packet size
one of the unit tests was failing because the openssh sftp server was dropping
the connection without any error. turns out they have a maximum allowed write
size (possibly around 64KB). the sftp rfcs have a small hint that some servers
may drop read/write requests of greater than 32KB.
so, all reads are limited to 32KB, and all writes > 32KB are now chopped up
and sent in 32KB chunks. this seems to keep openssh happy.
also, we now advertise 32KB max packet size instead of 8KB (the speed
improves a lot), and log when we read/write a packet. and sftp files are
flushed on seek.
speed up parts of BufferedFile
BufferedFile uses cStringIO for the write buffer now (i don't actually notice
any speed difference so this might revert later) and the default buffer size
has been upped from 1KB to 8KB.
when scanning for linefeeds (when writing to a line-buffered file), only scan
the newly-written bytes, since we know all the previously buffered data is
linefeed-free. this was the #1 slowdown on the 1MB-file unit test.
also, limit the buffering on line-buffered files to whatever the default
buffer size is. there's no reason to buffer 1MB waiting for a linefeed.
some Channel fixes for max packet size & blocking on zero window
some clean-ups and fixes to channels:
* when send() is blocked on a zero-width window, check that the channel is
still open. this was causing some lockups.
* set a lower bound to the "maximum packet size" we accept from the remote
host. if they tell us anything less than 1KB, assume they meant 1KB. (it's
not reasonable to fragment below that.)
* leave a little padding instead of cutting right up to the maximum packet
size: some space will be taken up by protocol overhead.
* turn off some of the debug log lines unless "ultra_debug" is on (nobody
cares about the feed info)
more unit tests
add a unit test for sending a large (1MB) file with line buffering but no
linefeeds (this triggered several bugs and inefficiencies), and another test
to verify that the write buffer is flushed on seek.
add forward.py demo script; bump to gyarados
add a demo script to show how to do local port forwarding.
add gyarados to all the docs and bump the version number everywhere.
add an sftp unit test for making 100 files
create 100 files on the remote server, set their mode with chmod, then verify
that they're all there and contain the right data. valeriy is reporting that
sometimes he's getting stuck after 20 and though i'm not seeing it, i want to
add a test to try to pin it down.
add direct-tcpip ability to open_channel
open_channel can now be given a dest_addr and src_addr, which are filled in
if the channel type is "forwarded-tcpip" or "direct-tcpip". these channel
types are used in remote & local port forwarding, respectively. i've only
tested "direct-tcpip" but i think if one works, they both should work.
also fixed a bug in connect where it was still assuming the old meaning for
get_remove_server_key() (oops!) and changed the sense of a send() failure
from <= 0 to < 0 since it may be possible for send() to return 0 and it not
be an EOF error.
add note about utf8 encodings
add info to the README about what to do if python complains about missing
encodings. veleriy pogrebitskiy ran into this and had advice.
fix deadlock in closing a channel
closing a channel would enter an odd codepath where the lock was grabbed,
some stuff was done, then another function was called where the lock was
grabbed again. unfortunately python locks aren't monitors so this would
deadlock. instead, make the smaller function lock-free with an explicit
notice that you must be holding the lock before calling.
fix utf8, raise packet size, log exceptions, be more lax with sfp servers
explicitly import utf8 encodings for "freezing" (and also because not all
platforms come with utf8, apparently). raise the max acceptable packet size
to 8kB, cuz 2kB was too low. log exceptions at error level instead of debug
level. and don't reject older sftp servers.
fearow date and last-minute fixes
update release date of fearow to 23apr. fix channel._set_closed() to grab
the lock before notifying the in/out buffers that the channel is closed.
try roger's trick for finding the home folder on windows.
add set_keepalive()
add set_keepalive() to set an automatic keepalive mechanism. (while waiting
for a packet on a connection, we periodically check if it's time to send a
keepalive packet.)
add get_username() method for remembering who you auth'd as
add get_username() method for remembering who you auth'd as. also, fix these
bugs:
* "continue" auth response counted as a failure (in server mode).
* try to import 'logging' in py22 before falling back to the fake logger,
in case they have a backported version of 'logger'
* raise the right exception when told to read a private key from a file that
isn't a private key file
* tell channels to close when the transport dies
fix encrypted private key files
the random byte padding on private key files' BER data was confusing openssh,
so switch to null-byte padding, which is slightly less secure but works with
crappy old openssh. also, enforce the mode when writing the private key
file. we really really want it to be 0600. (python seems to ignore the
mode normally.)
support py22, more or less
add roger binns' patches for supporting python 2.2. i hedged a bit on the
logging stuff and just added some trickery to let logging be stubbed out for
python 2.2. this changed a lot of import statements but i managed to avoid
hacking at any of the existing logging.
socket timeouts are required for the threads to notice when they've been
deactivated. worked around it by using the 'select' module on py22.
also fixed the sftp unit tests to cope with a password-protected private key.
make get_remote_server_key() return a PKey object
a good suggestion from roger binns: make get_remote_server_key() just return
a pkey object instead of a tuple of strings. all the strings can be extracted
from the pkey object, as well as other potentially useful things.
add dss key generation too, and fix some bugs
added the ability to generate dss keys and write private dss key files,
similar to rsa. in the process, fixed a couple of bugs with ber encoding
and writing password-encrypted key files. the key has to be padded to the
iblock size of the cipher -- it's very difficult to determine how the others
do this, so i just add random bytes to the end.
fixed the simple demo to use Transport's (host, port) constructor for
simplicity, and fixed a bug where the standard demo's DSS login wouldn't
work.
also, move the common logfile setup crap into util so all the demos can just
call that one.
add global request mechanism
add transport.global_request() to make a global-style request (usually an
extension to the protocol -- like keepalives) and handle requests from the
remote host. incoming requests are now handled and responded to correctly,
which should make openssh-style keepalives work. (before, we would silently
ignore them, which was wrong.)
can now generate rsa keys (not dss yet)
added functionality to ber to create ber streams. added some common methods
to PKey to allow dumping the key to base64 (the format used by openssh for
public key files and host key lists), and a factory for creating a key from
a private key file, and a common way to save private keys. RSAKey luckily
didn't have to change that much.
also added a factory method to RSAKey to generate a new key.
fix some arcana in unpacking private keys
"!= type([])" is a pretty obscure way to say it. let's try "is not list"
which is a lot more readable.
(mostly this is a test to make sure tla is working okay on my laptop.)
include tests in manifest
include the tests in the manifest for dist, and remove some outdated notes in
NOTES about the exported API (this is doc'd wayyy better in epydoc now).
add unit tests
add unit tests for BufferedFile and SFTP (it's a start). remove the demo sftp
client because it was 99% copied from the other demos, which makes it kinda
confusing. the unit tests are a much better example.
finish up client sftp support
added 'stat' to SFTPFile and SFTP, documented 'open' and 'listdir', and added
'rmdir', 'lstat', 'symlink', 'chmod', 'chown', 'utime', 'readlink'.
turned off ultra debugging now that the unit tests are all working.
fix some docs and BufferedFile.readline
fix some documentation and fix readline()'s universal newline support to
always return strings ending with '\n', regardless of how they were in the
original file. (this is an obvious feature of python's universal newline
support that i somehow missed before.)
fix lingering thread bug
this bug has been in there forever and i could never figure out a workaround
till now.
when the python interpreter exits, it doesn't necessarily destroy the
remaining objects or call __del__ on anything, and it will lock up until all
threads finish running. how the threads are supposed to notice the exiting
interpreter has always been sort of a mystery to me.
tonight i figured out how to use the 'atexit' module to register a handler
that runs when the interpreter exits. now we keep a list of active threads
and ask them all to exit on shutdown. no more going to another shell to
kill -9 python! yeah!!
add BufferedFile abstraction
SFTP client mode is mostly functional. there are probably still some bugs
but most of the operations on "file" objects have survived my simple tests.
BufferedFile wraps a simpler stream in something that looks like a python
file (and can even handle seeking if the stream underneath supports it).
it's meant to be subclassed. most of it is ripped out of what used to be
ChannelFile so i can reuse it for sftp -- ChannelFile is now tiny.
SFTP and Message are now exported.
fixed util.format_binary_line to not quote spaces.
Transport constructor can take hostname or address tuple
part of an ongoing attempt to make "simple" versions of some of the API calls,
so you can do common-case operations with just a few calls:
Transport's constructor will now let you pass in a string or tuple instead
of a socket-like object. if you pass in a string, it assumes the string is
a hostname (with optional ":port" segment) and turns that into an address
tuple. if you pass in a tuple, it assumes it's an address tuple. in both
cases, it then creates a socket, connects to the given address, and then
continues as if that was the socket passed in.
the idea being that you can call Transport('example.com') and it will do
the right thing.
pkey no longer raises binascii.Error
catch binascii.Error in the private key decoder and convert it into an
SSHException. there's no reason people should have to care that it was a
decoding error vs. any of the other million things that could be wrong in
a corrupt key file.
document more of Message; add get_int64
all of the get_* methods are now documented, but there's a bit more to do.
get_int64 added for eventual sftp support.
Robey Pointer