From: Mike Bayer Date: Fri, 21 Dec 2018 17:13:13 +0000 (-0500) Subject: Rewrite the convert_unicode docs and add tons of dragons X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=c3be61261b8e1050ca1ed5a9572bfbbfd0af5a23;p=thirdparty%2Fsqlalchemy%2Fsqlalchemy.git Rewrite the convert_unicode docs and add tons of dragons These flags will all be going away as Python 3 has solved all of this. Change-Id: I4f581d8dd7826dd823b671d0d8e72250284236c8 (cherry picked from commit 7f12f63c3a8c77715135126aac88ac6055b279c0) --- diff --git a/lib/sqlalchemy/engine/__init__.py b/lib/sqlalchemy/engine/__init__.py index ba74f37eee..d1a9382925 100644 --- a/lib/sqlalchemy/engine/__init__.py +++ b/lib/sqlalchemy/engine/__init__.py @@ -156,20 +156,45 @@ def create_engine(*args, **kwargs): additional keyword arguments. See the example at :ref:`custom_dbapi_args`. - :param convert_unicode=False: if set to True, sets - the default behavior of ``convert_unicode`` on the - :class:`.String` type to ``True``, regardless - of a setting of ``False`` on an individual - :class:`.String` type, thus causing all :class:`.String` - -based columns - to accommodate Python ``unicode`` objects. This flag - is useful as an engine-wide setting when using a - DBAPI that does not natively support Python - ``unicode`` objects and raises an error when - one is received (such as pyodbc with FreeTDS). - - See :class:`.String` for further details on - what this flag indicates. + :param convert_unicode=False: if set to True, causes + all :class:`.String` datatypes to act as though the + :paramref:`.String.convert_unicode` flag has been set to ``True``, + regardless of a setting of ``False`` on an individual :class:`.String` + type. This has the effect of causing all :class:`.String` -based + columns to accommodate Python Unicode objects directly as though the + datatype were the :class:`.Unicode` type. + + .. note:: + + SQLAlchemy's unicode-conversion flags and features only apply + to Python 2; in Python 3, all string objects are Unicode objects. + For this reason, as well as the fact that virtually all modern + DBAPIs now support Unicode natively even under Python 2, + the :paramref:`.Engine.convert_unicode` flag is inherently a + legacy feature. + + .. note:: + + This flag does **not** imply that SQLAlchemy's unicode-conversion + services will be used, as all modern DBAPIs already handle + unicode natively; in most cases it only indicates that the + :class:`.String` datatype will return Python unicode objects, + rather than plain strings. The :class:`.String` datatype itself + has additional options to force the usage of SQLAlchemy's unicode + converters. + + .. note:: + + This flag does **not** impact "raw" SQL statements that have no + typing information set up; that is, if the :class:`.String` + datatype is not used, no unicode behavior is implied. + + .. seealso:: + + :paramref:`.String.convert_unicode` - the flag local to the + :class:`.String` datatype has additional options + which can force unicode handling on a per-type basis. + :param creator: a callable which returns a DBAPI connection. This creation function will be passed to the underlying diff --git a/lib/sqlalchemy/sql/sqltypes.py b/lib/sqlalchemy/sql/sqltypes.py index 62c9b35df2..bd9c426261 100644 --- a/lib/sqlalchemy/sql/sqltypes.py +++ b/lib/sqlalchemy/sql/sqltypes.py @@ -153,35 +153,49 @@ class String(Concatenable, TypeEngine): :param convert_unicode: When set to ``True``, the :class:`.String` type will assume that - input is to be passed as Python ``unicode`` objects, - and results returned as Python ``unicode`` objects. - If the DBAPI in use does not support Python unicode - (which is fewer and fewer these days), SQLAlchemy - will encode/decode the value, using the - value of the ``encoding`` parameter passed to - :func:`.create_engine` as the encoding. - - When using a DBAPI that natively supports Python - unicode objects, this flag generally does not - need to be set. For columns that are explicitly - intended to store non-ASCII data, the :class:`.Unicode` - or :class:`.UnicodeText` - types should be used regardless, which feature - the same behavior of ``convert_unicode`` but - also indicate an underlying column type that - directly supports unicode, such as ``NVARCHAR``. - - For the extremely rare case that Python ``unicode`` + input is to be passed as Python Unicode objects under Python 2, + and results returned as Python Unicode objects. + In the rare circumstance that the DBAPI does not support + Python unicode under Python 2, SQLAlchemy will use its own + encoder/decoder functionality on strings, referring to the + value of the :paramref:`.create_engine.encoding` parameter + parameter passed to :func:`.create_engine` as the encoding. + + For the extremely rare case that Python Unicode is to be encoded/decoded by SQLAlchemy on a backend - that does natively support Python ``unicode``, - the value ``force`` can be passed here which will + that *does* natively support Python Unicode, + the string value ``"force"`` can be passed here which will cause SQLAlchemy's encode/decode services to be used unconditionally. + .. note:: + + SQLAlchemy's unicode-conversion flags and features only apply + to Python 2; in Python 3, all string objects are Unicode objects. + For this reason, as well as the fact that virtually all modern + DBAPIs now support Unicode natively even under Python 2, + the :paramref:`.String.convert_unicode` flag is inherently a + legacy feature. + + .. note:: + + In the vast majority of cases, the :class:`.Unicode` or + :class:`.UnicodeText` datatypes should be used for a + :class:`.Column` that expects to store non-ascii data. These + datatypes will ensure that the correct types are used on the + database side as well as set up the correct Unicode behaviors + under Python 2. + + .. seealso:: + + :paramref:`.create_engine.convert_unicode` - + :class:`.Engine`-wide parameter + :param unicode_error: Optional, a method to use to handle Unicode conversion errors. Behaves like the ``errors`` keyword argument to the standard library's ``string.decode()`` functions. This flag - requires that ``convert_unicode`` is set to ``force`` - otherwise, + requires that :paramref:`.String.convert_unicode` is set to + ``"force"`` - otherwise, SQLAlchemy is not guaranteed to handle the task of unicode conversion. Note that this flag adds significant performance overhead to row-fetching operations for backends that already