Mike Bayer [Thu, 6 May 2021 16:51:30 +0000 (12:51 -0400)]
narrow rangenotancestor to exclude target heads already present
Fixed regression in new revisioning traversal where "alembic downgrade
base" would fail if the database itself were clean and unversioned;
additionally repairs the case where downgrade would fail if attempting
to downgrade to the current head that is already present.
Mike Bayer [Tue, 27 Apr 2021 15:14:26 +0000 (11:14 -0400)]
Sort per-column comparisons in metadata order
Improved the rendering of ``op.add_column()`` operations when adding
multiple columns to an existing table, so that the order of these
statements matches the order in which the columns were declared in the
application's table metadata. Previously the added columns were being
sorted alphabetically.
Simon Bowly [Wed, 3 Feb 2021 03:11:15 +0000 (22:11 -0500)]
New downgrade algorithm to fix branch behaviour
The algorithm used for calculating downgrades/upgrades/iterating
revisions has been rewritten, to resolve ongoing issues of branches
not being handled consistently particularly within downgrade operations,
as well as for overall clarity and maintainability. This change includes
that a deprecation warning is emitted if an ambiguous command such
as "downgrade -1" when multiple heads are present is given.
In particular, the change implements a long-requested use case of allowing
downgrades of a single branch to a branchpoint.
Huge thanks to Simon Bowly for their impressive efforts in successfully
tackling this very difficult problem.
Topological algorithm written by Mike which retains the
behavior of walking along a single branch as long as
possible before switching to another branch due to
cross-dependencies.
Mike Bayer [Sat, 17 Apr 2021 16:53:22 +0000 (12:53 -0400)]
Ensure proxy transaction still present on exit before closing
Fixed regression caused by the SQLAlchemy 1.4/2.0 compatibility switch
where calling ``.rollback()`` or ``.commit()`` explicitly within the
``context.begin_transaction()`` context manager would cause it to fail when
the block ended, as it did not expect that the transaction was manually
closed.
Mike Bayer [Fri, 26 Feb 2021 18:11:17 +0000 (13:11 -0500)]
Create schema objects fresh from ops
Refactored the implementation of :class:`.MigrateOperation` constructs such
as :class:`.CreateIndexOp`, :class:`.CreateTableOp`, etc. so that they no
longer rely upon maintaining a persistent version of each schema object
internally; instead, the state variables of each operation object will be
used to produce the corresponding construct when the operation is invoked.
The rationale is so that environments which make use of
operation-manipulation schemes such as those those discussed in
:ref:`autogen_rewriter` are better supported, allowing end-user code to
manipulate the public attributes of these objects which will then be
expressed in the final output, an example is
``some_create_index_op.kw["postgresql_concurrently"] = True``.
Previously, these objects when generated from autogenerate would typically
hold onto the original, reflected element internally without honoring the
other state variables of each construct, preventing the public API from
working.
This feature is illustrated by the last line above.
### Description
* Improve the names of some variables recently created in #820 in order to make the code more readable.
* Add `cwd` as an option which is passed directly into `subprocess.run()`
* Add a brief description to the documentation, together with an example with `pre-commit`.
<!-- Describe your changes in detail -->
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
A new feature implementation
- [X] please include the issue number, and create an issue if none exists, which must
- [X] include a complete example of how the feature would look.
- [X] Please include: `Fixes: #<issue number>` in the commit message
- [x] **please include tests.**
**Help requested:** I can't think of any simple way to write a test. Does anyone have a suggestion???
Ben Mares [Sat, 27 Mar 2021 08:44:49 +0000 (04:44 -0400)]
Add REVISION_SCRIPT_FILENAME token for post_write_hooks options
I include a token called `REVISION_SCRIPT_FILENAME` which can be used in the `options` setting of a post-write hook. This token is replaced by the filename of the revision script, allowing the filename to be passed to the post-write hook in a flexible manner. Thus I can do:
Note that the existing behavior is to prepend `REVISION_SCRIPT_FILENAME` to the argument list. In order to preserve backwards compatibility, this is done when `REVISION_SCRIPT_FILENAME` is not present.
### Description
* Implement the `REVISION_SCRIPT_FILENAME` token.
* Switch from `str.split()` to `shlex.split()` for robust command line argument processing.
* Insert `REVISION_SCRIPT_FILENAME` in appropriate locations in the docs and templates.
* Properly document that `REVISION_SCRIPT_FILENAME` is prepended instead of appended to the argument list.
* Refactored existing "test of `black` as a post-write hook" in order to avoid code duplication.
* Add a test for `REVISION_SCRIPT_FILENAME` and shlex.
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [X] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Oliver Rice [Fri, 5 Mar 2021 19:23:40 +0000 (14:23 -0500)]
enable extensions to use AutogenContext.run_name_filters
Adjusted the recently added
:paramref:`.EnvironmentContext.configure.include_name` hook to accommodate
for additional object types such as "views" that don't have a parent table,
to support third party recipes and extensions. Pull request courtesy Oliver
Rice.
Mike Bayer [Thu, 4 Mar 2021 15:54:03 +0000 (10:54 -0500)]
Adjust mssql accommodate existing_type and type
Fixed bug where the "existing_type" parameter, which the MSSQL dialect
requires in order to change the nullability of a column in the absence of
also changing the column type, would cause an ALTER COLUMN operation to
incorrectly render a second ALTER statement without the nullability if a
new type were also present, as the MSSQL-specific contract did not
anticipate all three of "nullability", "type_" and "existing_type" being
sent at the same time.
CaselIT [Fri, 26 Feb 2021 20:21:52 +0000 (21:21 +0100)]
Add async template to Alembic
Add async template to Alembic to bootstrap environments that use
async DBAPI. Updated the cookbook to include a migration guide
on how to adapt an existing enviroment for use with DBAPI drivers.
Mike Bayer [Sat, 20 Feb 2021 20:48:58 +0000 (15:48 -0500)]
Implement sys_path_prepend option
Added new config file option ``prepend_sys_path``, which is a series of
paths that will be prepended to sys.path; the default value in newly
generated alembic.ini files is ".". This fixes a long-standing issue
where for some reason running the alembic command line would not place the
local "." path in sys.path, meaning an application locally present in "."
and importable through normal channels, e.g. python interpreter, pytest,
etc. would not be located by Alembic, even though the ``env.py`` file is
loaded relative to the current path when ``alembic.ini`` contains a
relative path.
A fix for #753 was merged in November 2020 and released in 1.5.0 in January 2021.
This updates the references to #753 in cookbook.rst to note that a fix for #753 has been released.
Mike Bayer [Tue, 2 Feb 2021 05:18:48 +0000 (00:18 -0500)]
Adjust for immediate dependencies that are still ancestors
Fixed bug in versioning model where a downgrade across a revision with a
dependency on another branch, yet an ancestor is also dependent on that
branch, would produce an erroneous state in the alembic_version table,
making upgrades impossible without manually repairing the table.
Mike Bayer [Fri, 29 Jan 2021 01:10:33 +0000 (20:10 -0500)]
Remove obsolete SQLAlchemy pk issue workaround
Fixed issue where autogenerate rendering of ``op.alter_column()`` would
fail to include MySQL ``existing_nullable=False`` if the column were part
of a primary key constraint within the table metadata.
Mike Bayer [Wed, 27 Jan 2021 17:28:37 +0000 (12:28 -0500)]
Place index/uq drops before creates
Changed the default ordering of "CREATE" and "DROP" statements indexes and
unique constraints within the autogenerate process, so that for example in
an upgrade() operation, a particular index or constraint that is to be
replaced such as for a casing convention change will not produce any naming
conflicts. For foreign key constraint objects, this is already how
constraints are ordered, and for table objects, users would normally want
to use :meth:`.Operations.rename_table` in any case.
Mike Bayer [Wed, 20 Jan 2021 16:40:52 +0000 (11:40 -0500)]
ensure downrev/dependencies in tuple form before using "in"
Fixed regression where new "loop detection" feature introduced in
:ticket:`757` produced false positives for revision names that have
overlapping substrings between revision number and down revision and/or
dependency, if the downrev/dependency were not in sequence form.
Mike Bayer [Tue, 19 Jan 2021 19:26:46 +0000 (14:26 -0500)]
retrieve 1.3 transaction from branched connection properly
Fixed regression where Alembic would fail to create a transaction properly
if the :class:`sqlalchemy.engine.Connection` were a so-called "branched"
connection, that is, one where the ``.connect()`` method had been called to
create a "sub" connection.
Mike Bayer [Mon, 18 Jan 2021 17:27:38 +0000 (12:27 -0500)]
implement include_name hook
Added new hook :paramref:`.EnvironmentContext.configure.include_name`,
which complements the
:paramref:`.EnvironmentContext.configure.include_object` hook by providing
a means of preventing objects of a certain name from being autogenerated
**before** the SQLAlchemy reflection process takes place, and notably
includes explicit support for passing each schema name when
:paramref:`.EnvironmentContext.configure.include_schemas` is set to True.
This is most important especially for enviroments that make use of
:paramref:`.EnvironmentContext.configure.include_schemas` where schemas are
actually databases (e.g. MySQL) in order to prevent reflection sweeps of
the entire server.
The long deprecated
:paramref:`.EnvironmentContext.configure.include_symbol` hook is removed.
The :paramref:`.EnvironmentContext.configure.include_object`
and :paramref:`.EnvironmentContext.configure.include_name`
hooks both achieve the goals of this hook.
Mike Bayer [Tue, 12 Jan 2021 21:33:31 +0000 (16:33 -0500)]
disable SQLA 1.4 asyncio mode
The latest merge that uses SQLAlchemy's test suite
more fully is not functioning with the asyncio wrappers.
As we don't need those here right now, disable them.
CaselIT [Mon, 19 Oct 2020 21:23:08 +0000 (23:23 +0200)]
Accommodate SQLAlchemy 1.4/2.0
To accommodate SQLAlchemy 1.4 and 2.0, the migration model now no longer
assumes that the SQLAlchemy Connection will autocommit an individual
operation. This essentially means that for databases that use
non-transactional DDL (pysqlite current driver behavior, MySQL), there is
still a BEGIN/COMMIT block that will surround each individual migration.
Databases that support transactional DDL should continue to have the
same flow, either per migration or per-entire run, depending on the
value of the :paramref:`.Environment.configure.transaction_per_migration`
flag.
Compatibility is established such that the entire library should
not generate any SQLAlchemy 2.0 deprecation warnings and
SQLALCHEMY_WARN_20 is part of conftest.py. (one warning remains
for the moment that needs to be resolved on the SQLAlchemy side)
The test suite requires SQLAlchemy 1.4.0b2 for testing 1.4;
1.4.0b1 won't work.
Test suite / setup also being modernized, as we are at
SQLAlchemy 1.3 we can now remove the majority of the testing
suite plugin.
Mike Bayer [Sun, 27 Dec 2020 21:40:48 +0000 (16:40 -0500)]
Consult ApplyBatchImpl for hints on constraints to drop
Made an adjustment to the PostgreSQL dialect to allow it to work more
effectively in batch mode, where a datatype like Boolean or non-native Enum
that may have embedded rules to generate CHECK constraints will be more
correctly handled in that these constraints usually will not have been
generated on the PostgreSQL backend; previously it would inadvertently
assume they existed unconditionally in a special PG-only "drop constraint"
step.
CaselIT [Tue, 17 Nov 2020 21:32:36 +0000 (22:32 +0100)]
Add support identity columns.
Added support for rendering of "identity" elements on
:class:`.Column` objects, supported in SQLAlchemy via
the :class:`.Identity` element introduced in version 1.4.
Adding columns with identity is supported on PostgreSQL,
MSSQL and Oracle. Changing the identity options or removing
it is supported only on PostgreSQL and Oracle.
Koichiro Den [Tue, 1 Dec 2020 02:53:33 +0000 (21:53 -0500)]
Raise an exception if any loop or cycle found in a revision graph constructed.
The revision tree is now checked for cycles and loops between revision
files when the revision environment is loaded up. Scenarios such as a
revision pointing to itself, or a revision that can reach itself via a
loop, are handled and will raise the :class:`.CycleDetected` exception when
the environment is loaded (expressed from the Alembic commandline as a
failure message and nonzero return code). Previously, these situations were
silently ignored up front, and the behavior of revision traversal would
either be silently incorrect, or would produce errors such as
:class:`.RangeNotAncestorError`. Pull request courtesy Koichiro Den.
Added detection of Table prefixes in the create table render
Added rendering for the ``Table.prefixes`` element to autogenerate so that
the rendered Python code includes these directives. Pull request courtesy
Rodrigo Ce Moretto.
Mike Bayer [Thu, 5 Nov 2020 13:22:18 +0000 (08:22 -0500)]
Copy column that's already attached
Modified the ``add_column()`` operation such that the ``Column`` object in
use is shallow copied to a new instance if that ``Column`` is already
attached to a ``table()`` or ``Table``. This accommodates for the change
made in SQLAlchemy issue #5618 which prohibits a ``Column`` from being
associated with multiple ``table()`` objects. This resumes support for
using a ``Column`` inside of an Alembic operation that already refers to a
parent ``table()`` or ``Table`` as well as allows operation objects just
autogenerated to work.
projects / thirdparty / sqlalchemy / alembic.git / log
