Merge branch '1.10' into sphinx-256

This commit is contained in:
Jeff Forcier 2014-02-14 12:10:02 -08:00
commit 6cdb8291b7
18 changed files with 126 additions and 155 deletions

1
.gitignore vendored
View File

@ -7,3 +7,4 @@ test.log
docs/
!sites/docs
_build
.coverage

View File

@ -8,7 +8,15 @@ install:
# Dev (doc/test running) requirements
- pip install coveralls # For coveralls.io specifically
- pip install -r dev-requirements.txt
script: coverage run --source=paramiko test.py --verbose
script:
# Main tests, with coverage!
- invoke coverage
# Ensure documentation & invoke pipeline run OK.
# Run 'docs' first since its objects.inv is referred to by 'www'.
# Also force warnings to be errors since most of them tend to be actual
# problems.
- invoke docs -o -W
- invoke www -o -W
notifications:
irc:
channels: "irc.freenode.org#paramiko"

98
NEWS
View File

@ -9,102 +9,14 @@ Issues noted as "'ssh' #NN" can be found at https://github.com/bitprophet/ssh/.
Issues noted as "Fabric #NN" can be found at https://github.com/fabric/fabric/.
**PLEASE NOTE:** For changes in 1.10.x and newer releases, please see
www.paramiko.org's changelog page, or the source file, sites/www/changelog.rst
Releases
========
v1.10.6 (21st Jan 2014)
-----------------------
* #193 (and its attentant PRs #230 & #253): Fix SSH agent problems present on
Windows. Thanks to David Hobbs for initial report and to Aarni Koskela & Olle
Lundberg for the patches.
v1.10.5 (8th Jan 2014)
----------------------
* #176: Fix AttributeError bugs in known_hosts file (re)loading. Thanks to
Nathan Scowcroft for the patch & Martin Blumenstingl for the initial test
case.
v1.10.4 (27th Sep 2013)
-----------------------
* #179: Fix a missing variable causing errors when an ssh_config file has a
non-default AddressFamily set. Thanks to Ed Marshall & Tomaz Muraus for catch
& patch.
* #200: Fix an exception-causing typo in `demo_simple.py`. Thanks to Alex
Buchanan for catch & Dave Foster for patch.
* #199: Typo fix in the license header cross-project. Thanks to Armin Ronacher
for catch & patch.
v1.10.3 (20th Sep 2013)
-----------------------
* #162: Clean up HMAC module import to avoid deadlocks in certain uses of
SSHClient. Thanks to Gernot Hillier for the catch & suggested
fix.
* #36: Fix the port-forwarding demo to avoid file descriptor errors. Thanks to
Jonathan Halcrow for catch & patch.
* #168: Update config handling to properly handle multiple 'localforward' and
'remoteforward' keys. Thanks to Emre Yılmaz for the patch.
v1.10.2 (26th Jul 2013)
-----------------------
* #153, #67: Warn on parse failure when reading known_hosts file. Thanks to
`@glasserc` for patch.
* #146: Indentation fixes for readability. Thanks to Abhinav Upadhyay for catch
& patch.
v1.10.1 (5th Apr 2013)
----------------------
* #142: (Fabric #811) SFTP put of empty file will still return the attributes
of the put file. Thanks to Jason R. Coombs for the patch.
* #154: (Fabric #876) Forwarded SSH agent connections left stale local pipes
lying around, which could cause local (and sometimes remote or network)
resource starvation when running many agent-using remote commands. Thanks to
Kevin Tegtmeier for catch & patch.
v1.10.0 (1st Mar 2013)
--------------------
* #66: Batch SFTP writes to help speed up file transfers. Thanks to Olle
Lundberg for the patch.
* #133: Fix handling of window-change events to be on-spec and not
attempt to wait for a response from the remote sshd; this fixes problems with
less common targets such as some Cisco devices. Thanks to Phillip Heller for
catch & patch.
* #93: Overhaul SSH config parsing to be in line with `man ssh_config` (& the
behavior of `ssh` itself), including addition of parameter expansion within
config values. Thanks to Olle Lundberg for the patch.
* #110: Honor SSH config `AddressFamily` setting when looking up local
host's FQDN. Thanks to John Hensley for the patch.
* #128: Defer FQDN resolution until needed, when parsing SSH config files.
Thanks to Parantapa Bhattacharya for catch & patch.
* #102: Forego random padding for packets when running under `*-ctr` ciphers.
This corrects some slowdowns on platforms where random byte generation is
inefficient (e.g. Windows). Thanks to `@warthog618` for catch & patch, and
Michael van der Kolff for code/technique review.
* #127: Turn `SFTPFile` into a context manager. Thanks to Michael Williamson
for the patch.
* #116: Limit `Message.get_bytes` to an upper bound of 1MB to protect against
potential DoS vectors. Thanks to `@mvschaik` for catch & patch.
* #115: Add convenience `get_pty` kwarg to `Client.exec_command` so users not
manually controlling a channel object can still toggle PTY creation. Thanks
to Michael van der Kolff for the patch.
* #71: Add `SFTPClient.putfo` and `.getfo` methods to allow direct
uploading/downloading of file-like objects. Thanks to Eric Buehl for the
patch.
* #113: Add `timeout` parameter to `SSHClient.exec_command` for easier setting
of the command's internal channel object's timeout. Thanks to Cernov Vladimir
for the patch.
* #94: Remove duplication of SSH port constant. Thanks to Olle Lundberg for the
catch.
* #80: Expose the internal "is closed" property of the file transfer class
`BufferedFile` as `.closed`, better conforming to Python's file interface.
Thanks to `@smunaut` and James Hiscock for catch & patch.
v1.9.0 (6th Nov 2012)
---------------------

View File

@ -1,7 +1,7 @@
# For newer tasks like building Sphinx docs.
# NOTE: Requires Python >=2.6
invoke>=0.6.1
invoke>=0.7.0
invocations>=0.4.4
sphinx>=1.1.3
alabaster>=0.1.0
releases>=0.2.4
alabaster>=0.3.0
releases>=0.5.1

View File

@ -55,7 +55,7 @@ if sys.version_info < (2, 5):
__author__ = "Jeff Forcier <jeff@bitprophet.org>"
__version__ = "1.10.5"
__version__ = "1.10.6"
__version_info__ = tuple([ int(d) for d in __version__.split(".") ])
__license__ = "GNU Lesser General Public License (LGPL)"

View File

@ -28,7 +28,7 @@ import UserDict
from paramiko.common import *
from paramiko.dsskey import DSSKey
from paramiko.rsakey import RSAKey
from paramiko.util import get_logger
from paramiko.util import get_logger, constant_time_bytes_eq
class InvalidHostKey(Exception):
@ -243,7 +243,7 @@ class HostKeys (UserDict.DictMixin):
entries = []
for e in self._entries:
for h in e.hostnames:
if (h.startswith('|1|') and (self.hash_host(hostname, h) == h)) or (h == hostname):
if h.startswith('|1|') and constant_time_bytes_eq(self.hash_host(hostname, h), h) or h == hostname:
entries.append(e)
if len(entries) == 0:
return None

View File

@ -358,7 +358,7 @@ class Packetizer (object):
mac = post_packet[:self.__mac_size_in]
mac_payload = struct.pack('>II', self.__sequence_number_in, packet_size) + packet
my_mac = compute_hmac(self.__mac_key_in, mac_payload, self.__mac_engine_in)[:self.__mac_size_in]
if my_mac != mac:
if not util.constant_time_bytes_eq(my_mac, mac):
raise SSHException('Mismatched MAC')
padding = ord(packet[0])
payload = packet[1:packet_size - padding]

View File

@ -736,7 +736,7 @@ class SFTPClient (BaseSFTP):
self._convert_status(msg)
return t, msg
if fileobj is not type(None):
fileobj._async_response(t, msg)
fileobj._async_response(t, msg, num)
if waitfor is None:
# just doing a single check
break

View File

@ -20,6 +20,8 @@
:class:`SFTPFile`
"""
from __future__ import with_statement
from binascii import hexlify
from collections import deque
import socket
@ -53,7 +55,8 @@ class SFTPFile (BufferedFile):
self._prefetching = False
self._prefetch_done = False
self._prefetch_data = {}
self._prefetch_reads = []
self._prefetch_extents = {}
self._prefetch_lock = threading.Lock()
self._saved_exception = None
self._reqs = deque()
@ -91,7 +94,7 @@ class SFTPFile (BufferedFile):
pass
def _data_in_prefetch_requests(self, offset, size):
k = [i for i in self._prefetch_reads if i[0] <= offset]
k = [x for x in self._prefetch_extents.values() if x[0] <= offset]
if len(k) == 0:
return False
k.sort(lambda x, y: cmp(x[0], y[0]))
@ -447,7 +450,6 @@ class SFTPFile (BufferedFile):
def _start_prefetch(self, chunks):
self._prefetching = True
self._prefetch_done = False
self._prefetch_reads.extend(chunks)
t = threading.Thread(target=self._prefetch_thread, args=(chunks,))
t.setDaemon(True)
@ -457,9 +459,11 @@ class SFTPFile (BufferedFile):
# do these read requests in a temporary thread because there may be
# a lot of them, so it may block.
for offset, length in chunks:
self.sftp._async_request(self, CMD_READ, self.handle, long(offset), int(length))
with self._prefetch_lock:
num = self.sftp._async_request(self, CMD_READ, self.handle, long(offset), int(length))
self._prefetch_extents[num] = (offset, length)
def _async_response(self, t, msg):
def _async_response(self, t, msg, num):
if t == CMD_STATUS:
# save exception and re-raise it on next file operation
try:
@ -470,10 +474,12 @@ class SFTPFile (BufferedFile):
if t != CMD_DATA:
raise SFTPError('Expected data')
data = msg.get_string()
offset, length = self._prefetch_reads.pop(0)
self._prefetch_data[offset] = data
if len(self._prefetch_reads) == 0:
self._prefetch_done = True
with self._prefetch_lock:
offset, length = self._prefetch_extents[num]
self._prefetch_data[offset] = data
del self._prefetch_extents[num]
if len(self._prefetch_extents) == 0:
self._prefetch_done = True
def _check_exception(self):
"if there's a saved exception, raise & clear it"

View File

@ -309,3 +309,12 @@ class Counter (object):
def new(cls, nbits, initial_value=1L, overflow=0L):
return cls(nbits, initial_value=initial_value, overflow=overflow)
new = classmethod(new)
def constant_time_bytes_eq(a, b):
if len(a) != len(b):
return False
res = 0
for i in xrange(len(a)):
res |= ord(a[i]) ^ ord(b[i])
return res == 0

View File

@ -52,7 +52,7 @@ if sys.platform == 'darwin':
setup(name = "paramiko",
version = "1.10.5",
version = "1.10.6",
description = "SSH2 protocol library",
author = "Jeff Forcier",
author_email = "jeff@bitprophet.org",

View File

@ -5,14 +5,13 @@ import sys
import alabaster
# Alabaster theme
# Alabaster theme + mini-extension
html_theme_path = [alabaster.get_path()]
extensions = ['alabaster']
# Paths relative to invoking conf.py - not this shared file
html_static_path = ['../_shared_static']
html_theme = 'alabaster'
html_theme_options = {
'logo': 'logo.png',
'logo_name': 'true',
'description': "A Python implementation of SSHv2.",
'github_user': 'paramiko',
'github_repo': 'paramiko',
@ -21,19 +20,11 @@ html_theme_options = {
'link': '#3782BE',
'link_hover': '#3782BE',
}
html_sidebars = {
# Landing page (no ToC)
'index': [
'about.html',
'searchbox.html',
'donate.html',
],
# Inner pages get a ToC
'**': [
'about.html',
'localtoc.html',
'navigation.html',
'searchbox.html',
'donate.html',
]
@ -42,7 +33,7 @@ html_sidebars = {
# Regular settings
project = u'Paramiko'
year = datetime.now().year
copyright = u'%d Jeff Forcier, 2003-2012 Robey Pointer' % year
copyright = u'%d Jeff Forcier' % year
master_doc = 'index'
templates_path = ['_templates']
exclude_trees = ['_build']

View File

@ -2,8 +2,15 @@
Changelog
=========
* :release:`1.10.6 <2014-01-21>`
* :bug:`193` (and its attentant PRs :issue:`230` & :issue:`253`): Fix SSH agent
* :bug:`-` Use constant-time hash comparison operations where possible, to
protect against `timing-based attacks
<http://codahale.com/a-lesson-in-timing-attacks/>`_. Thanks to Alex Gaynor
for the patch.
* :release:`1.10.6 <2014-02-14>`
* :bug:`34` (PR :issue:`35`) Fix SFTP prefetching incompatibility with some
SFTP servers regarding request/response ordering. Thanks to Richard
Kettlewell for catch & patch.
* :bug:`193` (and its attentant PRs :issue:`230` & :issue:`253`) Fix SSH agent
problems present on Windows. Thanks to David Hobbs for initial report and to
Aarni Koskela & Olle Lundberg for the patches.
* :release:`1.10.5 <2014-01-08>`
@ -14,7 +21,7 @@ Changelog
* :bug:`179` Fix a missing variable causing errors when an ssh_config file has
a non-default AddressFamily set. Thanks to Ed Marshall & Tomaz Muraus for
catch & patch.
* :bug:`200` Fix an exception-causing typo in `demo_simple.py`. Thanks to Alex
* :bug:`200` Fix an exception-causing typo in ``demo_simple.py``. Thanks to Alex
Buchanan for catch & Dave Foster for patch.
* :bug:`199` Typo fix in the license header cross-project. Thanks to Armin
Ronacher for catch & patch.
@ -27,7 +34,7 @@ Changelog
and 'remoteforward' keys. Thanks to Emre Yılmaz for the patch.
* :release:`1.10.2 <2013-07-26>`
* :bug:`153` (also :issue:`67`) Warn on parse failure when reading known_hosts
file. Thanks to `@glasserc` for patch.
file. Thanks to ``@glasserc`` for patch.
* :bug:`146` Indentation fixes for readability. Thanks to Abhinav Upadhyay for
catch & patch.
* :release:`1.10.1 <2013-04-05>`
@ -46,32 +53,32 @@ Changelog
attempt to wait for a response from the remote sshd; this fixes problems with
less common targets such as some Cisco devices. Thanks to Phillip Heller for
catch & patch.
* :feature:`93` Overhaul SSH config parsing to be in line with `man
ssh_config` (& the behavior of `ssh` itself), including addition of parameter
* :feature:`93` Overhaul SSH config parsing to be in line with ``man
ssh_config`` (& the behavior of ``ssh`` itself), including addition of parameter
expansion within config values. Thanks to Olle Lundberg for the patch.
* :feature:`110` Honor SSH config `AddressFamily` setting when looking up
* :feature:`110` Honor SSH config ``AddressFamily`` setting when looking up
local host's FQDN. Thanks to John Hensley for the patch.
* :feature:`128` Defer FQDN resolution until needed, when parsing SSH config
files. Thanks to Parantapa Bhattacharya for catch & patch.
* :bug:`102 major` Forego random padding for packets when running under
`*-ctr` ciphers. This corrects some slowdowns on platforms where random byte
generation is inefficient (e.g. Windows). Thanks to `@warthog618` for catch
& patch, and Michael van der Kolff for code/technique review.
* :feature:`127` Turn `SFTPFile` into a context manager. Thanks to Michael
``*-ctr`` ciphers. This corrects some slowdowns on platforms where random
byte generation is inefficient (e.g. Windows). Thanks to ``@warthog618`` for
catch & patch, and Michael van der Kolff for code/technique review.
* :feature:`127` Turn ``SFTPFile`` into a context manager. Thanks to Michael
Williamson for the patch.
* :feature:`116` Limit `Message.get_bytes` to an upper bound of 1MB to protect
against potential DoS vectors. Thanks to `@mvschaik` for catch & patch.
* :feature:`115` Add convenience `get_pty` kwarg to `Client.exec_command` so
* :feature:`116` Limit ``Message.get_bytes`` to an upper bound of 1MB to protect
against potential DoS vectors. Thanks to ``@mvschaik`` for catch & patch.
* :feature:`115` Add convenience ``get_pty`` kwarg to ``Client.exec_command`` so
users not manually controlling a channel object can still toggle PTY
creation. Thanks to Michael van der Kolff for the patch.
* :feature:`71` Add `SFTPClient.putfo` and `.getfo` methods to allow direct
* :feature:`71` Add ``SFTPClient.putfo`` and ``.getfo`` methods to allow direct
uploading/downloading of file-like objects. Thanks to Eric Buehl for the
patch.
* :feature:`113` Add `timeout` parameter to `SSHClient.exec_command` for
* :feature:`113` Add ``timeout`` parameter to ``SSHClient.exec_command`` for
easier setting of the command's internal channel object's timeout. Thanks to
Cernov Vladimir for the patch.
* :support:`94` Remove duplication of SSH port constant. Thanks to Olle
Lundberg for the catch.
* :feature:`80` Expose the internal "is closed" property of the file transfer
class `BufferedFile` as `.closed`, better conforming to Python's file
interface. Thanks to `@smunaut` and James Hiscock for catch & patch.
class ``BufferedFile`` as ``.closed``, better conforming to Python's file
interface. Thanks to ``@smunaut`` and James Hiscock for catch & patch.

View File

@ -1,15 +1,35 @@
# Obtain shared config values
import os, sys
sys.path.append(os.path.abspath('..'))
import sys
import os
from os.path import abspath, join, dirname
sys.path.append(abspath(join(dirname(__file__), '..')))
from shared_conf import *
# Add local blog extension
sys.path.append(os.path.abspath('.'))
extensions = ['blog']
# Local blog extension
sys.path.append(abspath('.'))
extensions.append('blog')
rss_link = 'http://paramiko.org'
rss_description = 'Paramiko project news'
# Add Releases changelog extension
# Releases changelog extension
extensions.append('releases')
releases_release_uri = "https://github.com/paramiko/paramiko/tree/%s"
releases_issue_uri = "https://github.com/paramiko/paramiko/issues/%s"
# Intersphinx for referencing API/usage docs
extensions.append('sphinx.ext.intersphinx')
# Default is 'local' building, but reference the public docs site when building
# under RTD.
target = join(dirname(__file__), '..', 'docs', '_build')
if os.environ.get('READTHEDOCS') == 'True':
# TODO: switch to docs.paramiko.org post go-live of sphinx API docs
target = 'http://paramiko-docs.readthedocs.org/en/latest/'
#intersphinx_mapping = {
# 'docs': (target, None),
#}
# Sister-site links to API docs
html_theme_options['extra_nav_links'] = {
"API Docs": 'http://docs.paramiko.org',
}

View File

@ -8,4 +8,4 @@ following ways:
* IRC: ``#paramiko`` on Freenode
* Mailing list: ``paramiko@librelist.com`` (see `the LibreList homepage
<http://librelist.com>`_ for usage details).
* This website's :doc:`blog </blog>`.
* This website - a blog section is forthcoming.

View File

@ -6,7 +6,7 @@ How to get the code
===================
Our primary Git repository is on Github at `paramiko/paramiko
<https://github.com/paramiko/paramiko>`; please follow their instruction for
<https://github.com/paramiko/paramiko>`_; please follow their instructions for
cloning to your local system. (If you intend to submit patches/pull requests,
we recommend forking first, then cloning your fork. Github has excellent
documentation for all this.)

View File

@ -12,12 +12,18 @@ usage and API documentation can be found at our code documentation site,
`docs.paramiko.org <http://docs.paramiko.org>`_.
.. toctree::
blog
changelog
installing
contributing
contact
.. Hide blog in hidden toctree for now (to avoid warnings.)
.. toctree::
:hidden:
blog
.. rubric:: Footnotes

View File

@ -1,7 +1,7 @@
from os.path import join
from invoke import Collection
from invocations import docs as _docs, testing
from invoke import Collection, ctask as task
from invocations import docs as _docs
d = 'sites'
@ -20,4 +20,15 @@ www = Collection.from_module(_docs, name='www', config={
'sphinx.target': join(path, '_build'),
})
ns = Collection(testing.test, docs=docs, www=www)
# Until we move to spec-based testing
@task
def test(ctx):
ctx.run("python test.py --verbose")
@task
def coverage(ctx):
ctx.run("coverage run --source=paramiko test.py --verbose")
ns = Collection(test, coverage, docs=docs, www=www)