Steven Bronson [Thu, 19 Aug 2021 16:06:25 +0000 (12:06 -0400)]
Fix postgresql_include in create_index
Fixed issue where usage of the PostgreSQL ``postgresql_include`` option
within a :meth:`.Operations.create_index` would raise a KeyError, as the
additional column(s) need to be added to the table object used by the
construct internally. The issue is equivalent to the SQL Server issue fixed
in :ticket:`513`. Pull request courtesy Steven Bronson.
Mike Bayer [Wed, 11 Aug 2021 19:58:16 +0000 (15:58 -0400)]
generate .pyi files for proxied classes
Stub .pyi files have been added for the "dynamically"
generated Alembic modules ``alembic.op`` and ``alembic.context``, which
include complete function signatures and docstrings, so that the functions
in these namespaces will have both IDE support (vscode, pycharm, etc) as
well as support for typing tools like Mypy. The files themselves are
statically generated from their source functions within the source tree.
Still not available is sphinx autodoc from the .pyi file. so
we might want to further modify this to write into the .py directly.
if we do *that*, we could almost get rid of ModuleClsProxy and
just generate proxying methods fully in op.py. however, this won't work
for the "extending ops" use case.
pep-484 type annotations have been added throughout the library. This
should be helpful in providing Mypy and IDE support, however there is not
full support for Alembic's dynamically modified "op" namespace as of yet; a
future release will likely modify the approach used for importing this
namespace to be better compatible with pep-484 capabilities.
Type originally created using MonkeyType
Add types extracted with the MonkeyType https://github.com/instagram/MonkeyType
library by running the unit tests using ``monkeytype run -m pytest tests``, then
``monkeytype apply <module>`` (see below for further details).
USed MonkeyType version 20.5 on Python 3.8, since newer version have issues
After applying the types, the new imports are placed in a ``TYPE_CHECKING`` guard
and all type definition of non base types are deferred by using the string notation.
NOTE: since to apply the types MonkeType need to import the module, also the test
ones, the patch below mocks the setup done by pytest so that the tests could be
correctly imported
diff --git a/alembic/testing/__init__.py b/alembic/testing/__init__.py
index bdd1746..b1090c7 100644
Change-Id: Iff93628f4b43c740848871ce077a118db5e75d41
--- a/alembic/testing/__init__.py
+++ b/alembic/testing/__init__.py
@@ -9,6 +9,12 @@ from sqlalchemy.testing.config import combinations
from sqlalchemy.testing.config import fixture
from sqlalchemy.testing.config import requirements as requires
+from sqlalchemy.testing.plugin.pytestplugin import PytestFixtureFunctions
+from sqlalchemy.testing.plugin.plugin_base import _setup_requirements
+
+config._fixture_functions = PytestFixtureFunctions()
+_setup_requirements("tests.requirements:DefaultRequirements")
+
from alembic import util
from .assertions import assert_raises
from .assertions import assert_raises_message
Currently I'm using this branch of the sqlalchemy stubs:
https://github.com/sqlalchemy/sqlalchemy2-stubs/tree/alembic_updates
Mike Bayer [Sat, 7 Aug 2021 15:48:19 +0000 (11:48 -0400)]
qualify sqlite batch add column for dynamic defaults
Batch "auto" mode will now select for "recreate" if the ``add_column()``
operation is used on SQLite, and the column itself meets the criteria for
SQLite where ADD COLUMN is not allowed, in this case a functional or
parenthesized SQL expression or a ``Computed`` (i.e. generated) column.
Nicolas CANIART [Tue, 10 Aug 2021 14:36:55 +0000 (10:36 -0400)]
Preserve comment and info in Drop/CreateTableOp
Fixed regression due to :ticket:`803` where the ``.info`` and ``.comment``
attributes of ``Table`` would be lost inside of the :class:`.DropTableOp`
class, which when "reversed" into a :class:`.CreateTableOp` would then have
lost these elements. Pull request courtesy Nicolas CANIART.
CaselIT [Tue, 15 Jun 2021 20:32:05 +0000 (22:32 +0200)]
Revendor editor and make dateutil optional
Re-implemented the ``python-editor`` dependency as a small internal
function to avoid the need for external dependencies.
The implementation is based on the original
version in 7b91b325ff43a0e9235e0f15b57391fa92352626.
Make the ``python-dateutil`` library an optional dependency.
This library is only required if the ``timezone`` option
is used in the Alembic configuration.
An extra require named ``tz`` is available with
``pip install alembic[tz]`` to install it.
Jet Zhou [Thu, 27 May 2021 18:26:51 +0000 (14:26 -0400)]
Add kwargs support to DropIndexOp autogenerate render
Fixed issue where dialect-specific keyword arguments within the
:class:`.DropIndex` operation directive would not render in the
autogenerated Python code. As support was improved for adding dialect
specific arguments to directives as part of :ticket:`803`, in particular
arguments such as "postgresql_concurrently" which apply to the actual
create/drop of the index, support was needed for these to render even in a
drop index operation. Pull request courtesy Jet Zhou.
Mike Bayer [Mon, 24 May 2021 20:14:30 +0000 (16:14 -0400)]
differentiate CreateTableOp from model vs. op.create_table()
Fixed regression caused by just fixed :ticket:`844` that scaled back the
filter for ``unique=True/index=True`` too far such that these directives no
longer worked for the ``op.create_table()`` op, this has been fixed.
Simon Bowly [Thu, 20 May 2021 21:46:04 +0000 (07:46 +1000)]
De-duplicate Revision.down_revision helpers.
Fixes an issue when downgrading a revision where dependencies
are duplicated (e.g. by specifying the down_revision in depends_on).
This causes the down migrations to fail to remove the head, which is
caught by an assertion.
Added logic to de-duplicate the entries in _normalized_down_revisions
and _all_down_revisions without altering revision order.
This change revises the previous approach taken in 8d5a9a1b0d32fff5726010afffa48cc0fb738238 to address the problem
more generally outside of the topological algorithm.
Mike Bayer [Thu, 20 May 2021 17:06:52 +0000 (13:06 -0400)]
Remove unique/index flags when copying table
Fixed 1.6-series regression where ``UniqueConstraint`` and to a lesser
extent ``Index`` objects would be doubled up in the generated model when
the ``unique=True`` / ``index=True`` flags were used.
Mike Bayer [Thu, 6 May 2021 16:51:30 +0000 (12:51 -0400)]
narrow rangenotancestor to exclude target heads already present
Fixed regression in new revisioning traversal where "alembic downgrade
base" would fail if the database itself were clean and unversioned;
additionally repairs the case where downgrade would fail if attempting
to downgrade to the current head that is already present.
Mike Bayer [Tue, 27 Apr 2021 15:14:26 +0000 (11:14 -0400)]
Sort per-column comparisons in metadata order
Improved the rendering of ``op.add_column()`` operations when adding
multiple columns to an existing table, so that the order of these
statements matches the order in which the columns were declared in the
application's table metadata. Previously the added columns were being
sorted alphabetically.
Simon Bowly [Wed, 3 Feb 2021 03:11:15 +0000 (22:11 -0500)]
New downgrade algorithm to fix branch behaviour
The algorithm used for calculating downgrades/upgrades/iterating
revisions has been rewritten, to resolve ongoing issues of branches
not being handled consistently particularly within downgrade operations,
as well as for overall clarity and maintainability. This change includes
that a deprecation warning is emitted if an ambiguous command such
as "downgrade -1" when multiple heads are present is given.
In particular, the change implements a long-requested use case of allowing
downgrades of a single branch to a branchpoint.
Huge thanks to Simon Bowly for their impressive efforts in successfully
tackling this very difficult problem.
Topological algorithm written by Mike which retains the
behavior of walking along a single branch as long as
possible before switching to another branch due to
cross-dependencies.
Mike Bayer [Sat, 17 Apr 2021 16:53:22 +0000 (12:53 -0400)]
Ensure proxy transaction still present on exit before closing
Fixed regression caused by the SQLAlchemy 1.4/2.0 compatibility switch
where calling ``.rollback()`` or ``.commit()`` explicitly within the
``context.begin_transaction()`` context manager would cause it to fail when
the block ended, as it did not expect that the transaction was manually
closed.
Mike Bayer [Fri, 26 Feb 2021 18:11:17 +0000 (13:11 -0500)]
Create schema objects fresh from ops
Refactored the implementation of :class:`.MigrateOperation` constructs such
as :class:`.CreateIndexOp`, :class:`.CreateTableOp`, etc. so that they no
longer rely upon maintaining a persistent version of each schema object
internally; instead, the state variables of each operation object will be
used to produce the corresponding construct when the operation is invoked.
The rationale is so that environments which make use of
operation-manipulation schemes such as those those discussed in
:ref:`autogen_rewriter` are better supported, allowing end-user code to
manipulate the public attributes of these objects which will then be
expressed in the final output, an example is
``some_create_index_op.kw["postgresql_concurrently"] = True``.
Previously, these objects when generated from autogenerate would typically
hold onto the original, reflected element internally without honoring the
other state variables of each construct, preventing the public API from
working.
This feature is illustrated by the last line above.
### Description
* Improve the names of some variables recently created in #820 in order to make the code more readable.
* Add `cwd` as an option which is passed directly into `subprocess.run()`
* Add a brief description to the documentation, together with an example with `pre-commit`.
<!-- Describe your changes in detail -->
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
A new feature implementation
- [X] please include the issue number, and create an issue if none exists, which must
- [X] include a complete example of how the feature would look.
- [X] Please include: `Fixes: #<issue number>` in the commit message
- [x] **please include tests.**
**Help requested:** I can't think of any simple way to write a test. Does anyone have a suggestion???
Ben Mares [Sat, 27 Mar 2021 08:44:49 +0000 (04:44 -0400)]
Add REVISION_SCRIPT_FILENAME token for post_write_hooks options
I include a token called `REVISION_SCRIPT_FILENAME` which can be used in the `options` setting of a post-write hook. This token is replaced by the filename of the revision script, allowing the filename to be passed to the post-write hook in a flexible manner. Thus I can do:
Note that the existing behavior is to prepend `REVISION_SCRIPT_FILENAME` to the argument list. In order to preserve backwards compatibility, this is done when `REVISION_SCRIPT_FILENAME` is not present.
### Description
* Implement the `REVISION_SCRIPT_FILENAME` token.
* Switch from `str.split()` to `shlex.split()` for robust command line argument processing.
* Insert `REVISION_SCRIPT_FILENAME` in appropriate locations in the docs and templates.
* Properly document that `REVISION_SCRIPT_FILENAME` is prepended instead of appended to the argument list.
* Refactored existing "test of `black` as a post-write hook" in order to avoid code duplication.
* Add a test for `REVISION_SCRIPT_FILENAME` and shlex.
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [X] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Oliver Rice [Fri, 5 Mar 2021 19:23:40 +0000 (14:23 -0500)]
enable extensions to use AutogenContext.run_name_filters
Adjusted the recently added
:paramref:`.EnvironmentContext.configure.include_name` hook to accommodate
for additional object types such as "views" that don't have a parent table,
to support third party recipes and extensions. Pull request courtesy Oliver
Rice.
Mike Bayer [Thu, 4 Mar 2021 15:54:03 +0000 (10:54 -0500)]
Adjust mssql accommodate existing_type and type
Fixed bug where the "existing_type" parameter, which the MSSQL dialect
requires in order to change the nullability of a column in the absence of
also changing the column type, would cause an ALTER COLUMN operation to
incorrectly render a second ALTER statement without the nullability if a
new type were also present, as the MSSQL-specific contract did not
anticipate all three of "nullability", "type_" and "existing_type" being
sent at the same time.
CaselIT [Fri, 26 Feb 2021 20:21:52 +0000 (21:21 +0100)]
Add async template to Alembic
Add async template to Alembic to bootstrap environments that use
async DBAPI. Updated the cookbook to include a migration guide
on how to adapt an existing enviroment for use with DBAPI drivers.
Mike Bayer [Sat, 20 Feb 2021 20:48:58 +0000 (15:48 -0500)]
Implement sys_path_prepend option
Added new config file option ``prepend_sys_path``, which is a series of
paths that will be prepended to sys.path; the default value in newly
generated alembic.ini files is ".". This fixes a long-standing issue
where for some reason running the alembic command line would not place the
local "." path in sys.path, meaning an application locally present in "."
and importable through normal channels, e.g. python interpreter, pytest,
etc. would not be located by Alembic, even though the ``env.py`` file is
loaded relative to the current path when ``alembic.ini`` contains a
relative path.
A fix for #753 was merged in November 2020 and released in 1.5.0 in January 2021.
This updates the references to #753 in cookbook.rst to note that a fix for #753 has been released.
Mike Bayer [Tue, 2 Feb 2021 05:18:48 +0000 (00:18 -0500)]
Adjust for immediate dependencies that are still ancestors
Fixed bug in versioning model where a downgrade across a revision with a
dependency on another branch, yet an ancestor is also dependent on that
branch, would produce an erroneous state in the alembic_version table,
making upgrades impossible without manually repairing the table.
Mike Bayer [Fri, 29 Jan 2021 01:10:33 +0000 (20:10 -0500)]
Remove obsolete SQLAlchemy pk issue workaround
Fixed issue where autogenerate rendering of ``op.alter_column()`` would
fail to include MySQL ``existing_nullable=False`` if the column were part
of a primary key constraint within the table metadata.
Mike Bayer [Wed, 27 Jan 2021 17:28:37 +0000 (12:28 -0500)]
Place index/uq drops before creates
Changed the default ordering of "CREATE" and "DROP" statements indexes and
unique constraints within the autogenerate process, so that for example in
an upgrade() operation, a particular index or constraint that is to be
replaced such as for a casing convention change will not produce any naming
conflicts. For foreign key constraint objects, this is already how
constraints are ordered, and for table objects, users would normally want
to use :meth:`.Operations.rename_table` in any case.
Mike Bayer [Wed, 20 Jan 2021 16:40:52 +0000 (11:40 -0500)]
ensure downrev/dependencies in tuple form before using "in"
Fixed regression where new "loop detection" feature introduced in
:ticket:`757` produced false positives for revision names that have
overlapping substrings between revision number and down revision and/or
dependency, if the downrev/dependency were not in sequence form.
projects / thirdparty / sqlalchemy / alembic.git / log
