Rewrite the convert_unicode docs and add tons of dragons

These flags will all be going away as Python 3 has solved
all of this.

Change-Id: I4f581d8dd7826dd823b671d0d8e72250284236c8
(cherry picked from commit 7f12f63c3a8c77715135126aac88ac6055b279c0)

diff --git a/lib/sqlalchemy/engine/__init__.py b/lib/sqlalchemy/engine/__init__.py
index ba74f37eeeaee52b564e4fee184a583345d32763..d1a93829255ee262a9edc8a481bdd94e2fa7e2a0 100644 (file)
--- a/lib/sqlalchemy/engine/__init__.py
+++ b/lib/sqlalchemy/engine/__init__.py
@@ -156,20 +156,45 @@ def create_engine(*args, **kwargs):
         additional keyword arguments.  See the example
         at :ref:`custom_dbapi_args`.
 
-    :param convert_unicode=False: if set to True, sets
-        the default behavior of ``convert_unicode`` on the
-        :class:`.String` type to ``True``, regardless
-        of a setting of ``False`` on an individual
-        :class:`.String` type, thus causing all :class:`.String`
-        -based columns
-        to accommodate Python ``unicode`` objects.  This flag
-        is useful as an engine-wide setting when using a
-        DBAPI that does not natively support Python
-        ``unicode`` objects and raises an error when
-        one is received (such as pyodbc with FreeTDS).
-
-        See :class:`.String` for further details on
-        what this flag indicates.
+    :param convert_unicode=False: if set to True, causes
+        all :class:`.String` datatypes to act as though the
+        :paramref:`.String.convert_unicode` flag has been set to ``True``,
+        regardless of a setting of ``False`` on an individual :class:`.String`
+        type.  This has the effect of causing all :class:`.String` -based
+        columns to accommodate Python Unicode objects directly as though the
+        datatype were the :class:`.Unicode` type.
+
+        .. note::
+
+            SQLAlchemy's unicode-conversion flags and features only apply
+            to Python 2; in Python 3, all string objects are Unicode objects.
+            For this reason, as well as the fact that virtually all modern
+            DBAPIs now support Unicode natively even under Python 2,
+            the :paramref:`.Engine.convert_unicode` flag is inherently a
+            legacy feature.
+
+        .. note::
+
+            This flag does **not** imply that SQLAlchemy's unicode-conversion
+            services will be used, as all modern DBAPIs already handle
+            unicode natively; in most cases it only indicates that the
+            :class:`.String` datatype will return Python unicode objects,
+            rather than plain strings.   The :class:`.String` datatype itself
+            has additional options to force the usage of SQLAlchemy's unicode
+            converters.
+
+        .. note::
+
+            This flag does **not** impact "raw" SQL statements that have no
+            typing information set up; that is, if the :class:`.String`
+            datatype is not used, no unicode behavior is implied.
+
+        .. seealso::
+
+            :paramref:`.String.convert_unicode` - the flag local to the
+            :class:`.String` datatype has additional options
+            which can force unicode handling on a per-type basis.
+
 
     :param creator: a callable which returns a DBAPI connection.
         This creation function will be passed to the underlying

diff --git a/lib/sqlalchemy/sql/sqltypes.py b/lib/sqlalchemy/sql/sqltypes.py
index 62c9b35df267617bdac9c61c6cab71d915252141..bd9c4262613899b0a4695c5b237d5930d53dd40d 100644 (file)
--- a/lib/sqlalchemy/sql/sqltypes.py
+++ b/lib/sqlalchemy/sql/sqltypes.py
@@ -153,35 +153,49 @@ class String(Concatenable, TypeEngine):
 
         :param convert_unicode: When set to ``True``, the
           :class:`.String` type will assume that
-          input is to be passed as Python ``unicode`` objects,
-          and results returned as Python ``unicode`` objects.
-          If the DBAPI in use does not support Python unicode
-          (which is fewer and fewer these days), SQLAlchemy
-          will encode/decode the value, using the
-          value of the ``encoding`` parameter passed to
-          :func:`.create_engine` as the encoding.
-
-          When using a DBAPI that natively supports Python
-          unicode objects, this flag generally does not
-          need to be set.  For columns that are explicitly
-          intended to store non-ASCII data, the :class:`.Unicode`
-          or :class:`.UnicodeText`
-          types should be used regardless, which feature
-          the same behavior of ``convert_unicode`` but
-          also indicate an underlying column type that
-          directly supports unicode, such as ``NVARCHAR``.
-
-          For the extremely rare case that Python ``unicode``
+          input is to be passed as Python Unicode objects under Python 2,
+          and results returned as Python Unicode objects.
+          In the rare circumstance that the DBAPI does not support
+          Python unicode under Python 2, SQLAlchemy will use its own
+          encoder/decoder functionality on strings, referring to the
+          value of the :paramref:`.create_engine.encoding` parameter
+          parameter passed to :func:`.create_engine` as the encoding.
+
+          For the extremely rare case that Python Unicode
           is to be encoded/decoded by SQLAlchemy on a backend
-          that does natively support Python ``unicode``,
-          the value ``force`` can be passed here which will
+          that *does* natively support Python Unicode,
+          the string value ``"force"`` can be passed here which will
           cause SQLAlchemy's encode/decode services to be
           used unconditionally.
 
+          .. note::
+
+            SQLAlchemy's unicode-conversion flags and features only apply
+            to Python 2; in Python 3, all string objects are Unicode objects.
+            For this reason, as well as the fact that virtually all modern
+            DBAPIs now support Unicode natively even under Python 2,
+            the :paramref:`.String.convert_unicode` flag is inherently a
+            legacy feature.
+
+          .. note::
+
+            In the vast majority of cases, the :class:`.Unicode` or
+            :class:`.UnicodeText` datatypes should be used for a
+            :class:`.Column` that expects to store non-ascii data.  These
+            datatypes will ensure that the correct types are used on the
+            database side as well as set up the correct Unicode behaviors
+            under Python 2.
+
+          .. seealso::
+
+            :paramref:`.create_engine.convert_unicode` -
+            :class:`.Engine`-wide parameter
+
         :param unicode_error: Optional, a method to use to handle Unicode
           conversion errors. Behaves like the ``errors`` keyword argument to
           the standard library's ``string.decode()`` functions.   This flag
-          requires that ``convert_unicode`` is set to ``force`` - otherwise,
+          requires that :paramref:`.String.convert_unicode` is set to
+          ``"force"`` - otherwise,
           SQLAlchemy is not guaranteed to handle the task of unicode
           conversion.   Note that this flag adds significant performance
           overhead to row-fetching operations for backends that already