add settimeout/gettimeout/setblocking, some bugfixes.
hide the command and response codes in sftp so they aren't exported.
add settimeout/gettimeout/setblocking that just wrap calls to the underlying
socket or channel. fix _read_all to not catch timeout exceptions.
limit read/write requests to 32KB, advertise 32KB max packet size
one of the unit tests was failing because the openssh sftp server was dropping
the connection without any error. turns out they have a maximum allowed write
size (possibly around 64KB). the sftp rfcs have a small hint that some servers
may drop read/write requests of greater than 32KB.
so, all reads are limited to 32KB, and all writes > 32KB are now chopped up
and sent in 32KB chunks. this seems to keep openssh happy.
also, we now advertise 32KB max packet size instead of 8KB (the speed
improves a lot), and log when we read/write a packet. and sftp files are
flushed on seek.
speed up parts of BufferedFile
BufferedFile uses cStringIO for the write buffer now (i don't actually notice
any speed difference so this might revert later) and the default buffer size
has been upped from 1KB to 8KB.
when scanning for linefeeds (when writing to a line-buffered file), only scan
the newly-written bytes, since we know all the previously buffered data is
linefeed-free. this was the #1 slowdown on the 1MB-file unit test.
also, limit the buffering on line-buffered files to whatever the default
buffer size is. there's no reason to buffer 1MB waiting for a linefeed.
some Channel fixes for max packet size & blocking on zero window
some clean-ups and fixes to channels:
* when send() is blocked on a zero-width window, check that the channel is
still open. this was causing some lockups.
* set a lower bound to the "maximum packet size" we accept from the remote
host. if they tell us anything less than 1KB, assume they meant 1KB. (it's
not reasonable to fragment below that.)
* leave a little padding instead of cutting right up to the maximum packet
size: some space will be taken up by protocol overhead.
* turn off some of the debug log lines unless "ultra_debug" is on (nobody
cares about the feed info)
more unit tests
add a unit test for sending a large (1MB) file with line buffering but no
linefeeds (this triggered several bugs and inefficiencies), and another test
to verify that the write buffer is flushed on seek.
add forward.py demo script; bump to gyarados
add a demo script to show how to do local port forwarding.
add gyarados to all the docs and bump the version number everywhere.
add an sftp unit test for making 100 files
create 100 files on the remote server, set their mode with chmod, then verify
that they're all there and contain the right data. valeriy is reporting that
sometimes he's getting stuck after 20 and though i'm not seeing it, i want to
add a test to try to pin it down.
add direct-tcpip ability to open_channel
open_channel can now be given a dest_addr and src_addr, which are filled in
if the channel type is "forwarded-tcpip" or "direct-tcpip". these channel
types are used in remote & local port forwarding, respectively. i've only
tested "direct-tcpip" but i think if one works, they both should work.
also fixed a bug in connect where it was still assuming the old meaning for
get_remove_server_key() (oops!) and changed the sense of a send() failure
from <= 0 to < 0 since it may be possible for send() to return 0 and it not
be an EOF error.
add note about utf8 encodings
add info to the README about what to do if python complains about missing
encodings. veleriy pogrebitskiy ran into this and had advice.
fix deadlock in closing a channel
closing a channel would enter an odd codepath where the lock was grabbed,
some stuff was done, then another function was called where the lock was
grabbed again. unfortunately python locks aren't monitors so this would
deadlock. instead, make the smaller function lock-free with an explicit
notice that you must be holding the lock before calling.
fix utf8, raise packet size, log exceptions, be more lax with sfp servers
explicitly import utf8 encodings for "freezing" (and also because not all
platforms come with utf8, apparently). raise the max acceptable packet size
to 8kB, cuz 2kB was too low. log exceptions at error level instead of debug
level. and don't reject older sftp servers.
fearow date and last-minute fixes
update release date of fearow to 23apr. fix channel._set_closed() to grab
the lock before notifying the in/out buffers that the channel is closed.
try roger's trick for finding the home folder on windows.
add set_keepalive()
add set_keepalive() to set an automatic keepalive mechanism. (while waiting
for a packet on a connection, we periodically check if it's time to send a
keepalive packet.)
add get_username() method for remembering who you auth'd as
add get_username() method for remembering who you auth'd as. also, fix these
bugs:
* "continue" auth response counted as a failure (in server mode).
* try to import 'logging' in py22 before falling back to the fake logger,
in case they have a backported version of 'logger'
* raise the right exception when told to read a private key from a file that
isn't a private key file
* tell channels to close when the transport dies
fix encrypted private key files
the random byte padding on private key files' BER data was confusing openssh,
so switch to null-byte padding, which is slightly less secure but works with
crappy old openssh. also, enforce the mode when writing the private key
file. we really really want it to be 0600. (python seems to ignore the
mode normally.)
support py22, more or less
add roger binns' patches for supporting python 2.2. i hedged a bit on the
logging stuff and just added some trickery to let logging be stubbed out for
python 2.2. this changed a lot of import statements but i managed to avoid
hacking at any of the existing logging.
socket timeouts are required for the threads to notice when they've been
deactivated. worked around it by using the 'select' module on py22.
also fixed the sftp unit tests to cope with a password-protected private key.
make get_remote_server_key() return a PKey object
a good suggestion from roger binns: make get_remote_server_key() just return
a pkey object instead of a tuple of strings. all the strings can be extracted
from the pkey object, as well as other potentially useful things.
add dss key generation too, and fix some bugs
added the ability to generate dss keys and write private dss key files,
similar to rsa. in the process, fixed a couple of bugs with ber encoding
and writing password-encrypted key files. the key has to be padded to the
iblock size of the cipher -- it's very difficult to determine how the others
do this, so i just add random bytes to the end.
fixed the simple demo to use Transport's (host, port) constructor for
simplicity, and fixed a bug where the standard demo's DSS login wouldn't
work.
also, move the common logfile setup crap into util so all the demos can just
call that one.
add global request mechanism
add transport.global_request() to make a global-style request (usually an
extension to the protocol -- like keepalives) and handle requests from the
remote host. incoming requests are now handled and responded to correctly,
which should make openssh-style keepalives work. (before, we would silently
ignore them, which was wrong.)
can now generate rsa keys (not dss yet)
added functionality to ber to create ber streams. added some common methods
to PKey to allow dumping the key to base64 (the format used by openssh for
public key files and host key lists), and a factory for creating a key from
a private key file, and a common way to save private keys. RSAKey luckily
didn't have to change that much.
also added a factory method to RSAKey to generate a new key.
fix some arcana in unpacking private keys
"!= type([])" is a pretty obscure way to say it. let's try "is not list"
which is a lot more readable.
(mostly this is a test to make sure tla is working okay on my laptop.)
include tests in manifest
include the tests in the manifest for dist, and remove some outdated notes in
NOTES about the exported API (this is doc'd wayyy better in epydoc now).
add unit tests
add unit tests for BufferedFile and SFTP (it's a start). remove the demo sftp
client because it was 99% copied from the other demos, which makes it kinda
confusing. the unit tests are a much better example.
finish up client sftp support
added 'stat' to SFTPFile and SFTP, documented 'open' and 'listdir', and added
'rmdir', 'lstat', 'symlink', 'chmod', 'chown', 'utime', 'readlink'.
turned off ultra debugging now that the unit tests are all working.
fix some docs and BufferedFile.readline
fix some documentation and fix readline()'s universal newline support to
always return strings ending with '\n', regardless of how they were in the
original file. (this is an obvious feature of python's universal newline
support that i somehow missed before.)
fix lingering thread bug
this bug has been in there forever and i could never figure out a workaround
till now.
when the python interpreter exits, it doesn't necessarily destroy the
remaining objects or call __del__ on anything, and it will lock up until all
threads finish running. how the threads are supposed to notice the exiting
interpreter has always been sort of a mystery to me.
tonight i figured out how to use the 'atexit' module to register a handler
that runs when the interpreter exits. now we keep a list of active threads
and ask them all to exit on shutdown. no more going to another shell to
kill -9 python! yeah!!
add BufferedFile abstraction
SFTP client mode is mostly functional. there are probably still some bugs
but most of the operations on "file" objects have survived my simple tests.
BufferedFile wraps a simpler stream in something that looks like a python
file (and can even handle seeking if the stream underneath supports it).
it's meant to be subclassed. most of it is ripped out of what used to be
ChannelFile so i can reuse it for sftp -- ChannelFile is now tiny.
SFTP and Message are now exported.
fixed util.format_binary_line to not quote spaces.
Transport constructor can take hostname or address tuple
part of an ongoing attempt to make "simple" versions of some of the API calls,
so you can do common-case operations with just a few calls:
Transport's constructor will now let you pass in a string or tuple instead
of a socket-like object. if you pass in a string, it assumes the string is
a hostname (with optional ":port" segment) and turns that into an address
tuple. if you pass in a tuple, it assumes it's an address tuple. in both
cases, it then creates a socket, connects to the given address, and then
continues as if that was the socket passed in.
the idea being that you can call Transport('example.com') and it will do
the right thing.
pkey no longer raises binascii.Error
catch binascii.Error in the private key decoder and convert it into an
SSHException. there's no reason people should have to care that it was a
decoding error vs. any of the other million things that could be wrong in
a corrupt key file.
document more of Message; add get_int64
all of the get_* methods are now documented, but there's a bit more to do.
get_int64 added for eventual sftp support.
fix MANIFEST.in, change version numbers to 0.9-doduo, fix LPGL notices
fixed MANIFEST.in to include the demo scripts, LICENSE, and ChangeLog.
upped everything to version 0.9-doduo.
fixed the copyright notice, and added the LGPL banner to the top of every
python file.
more docs, and password-protected key files can now be read
lots more documentation, some of it moved out of the README file, which is
now much smaller and less rambling.
repr(Transport) now reports the number of bits used in the cipher.
cleaned up BER to use util functions, and throw a proper exception (the new
BERException) on error. it doesn't ever have to be a full BER decoder, but
it can at least comb its hair and tuck in its shirt.
lots of stuff added to PKey.read_private_key_file so it can try to decode
password-protected key files. right now it only understands "DES-EDE3-CBC"
format, but this is the only format i've seen openssh make so far. if the
key is password-protected, but no password was given, a new exception
(PasswordRequiredException) is raised so an outer layer can ask for a password
and try again.
added public-key support to server mode, more docs
added public-key support to server mode (it can now verify a client signature)
and added a demo of that to the demo_server.py script (user_rsa_key). in the
process, cleaned up the API of PKey so that now it only has to know about
signing and verifying ssh2 blobs, and can be hashed and compared with other
keys (comparing & hashing only the public parts of the key). keys can also
be created from strings now too.
some more documentation and hiding private methods.
lots more documentation, and added Transport.connect()
renamed demo_host_key to demo_rsa_key. moved changelog to a separate file,
and indicated that future changelog entries should be fetched from tla.
tried to clean up "__all__" in a way that makes epydoc still work.
added lots more documentation, and renamed many methods and vars to hide
them as private non-exported API.
Transport's ModulusPack is now a static member, so it only has to be loaded
once, and can then be used by any future Transport object.
added Transport.connect(), which tries to wrap all the SSH2 negotiation and
authentication into one method. you should be able to create a Transport,
call connect(), and then create channels.
hook up server-side kex-gex; add more documentation
group-exchange kex should work now on the server side. it will only be
advertised if a "moduli" file has been loaded (see the -gasp- docs) so we
don't spend hours (literally. hours.) computing primes. some of the logic
was previously wrong, too, since it had never been tested.
fixed repr() string for Transport/BaseTransport. moved is_authenticated to
Transport where it belongs.
added lots of documentation (but still only about 10% documented). lots of
methods were made private finally.
fix up new paramiko/ folder.
moved SSHException to a new file (ssh_exception.py) and turned paramiko.py
into an __init__.py file. i'm still not entirely sure how this normally
works, so i may have done something wrong, but it's supposed to work the
same as before.
Robey Pointer