Mike Bayer [Mon, 19 May 2025 17:06:56 +0000 (13:06 -0400)]
use pathlib for file operations across the board
The command, config and script modules now rely on ``pathlib.Path`` for
internal path manipulations, instead of ``os.path()`` operations. Public
API functions that accept string directories and filenames continue to do
so but also accept ``os.PathLike`` objects. Public API functions and
accessors that return paths as strings continue to do so. Private API
functions and accessors, i.e. all those that are prefixed with an
underscore, may now return a Path object rather than a string to indicate
file paths.
Add support of IF [NOT] EXISTS for ADD/DROP COLUMN in Postgresql
Added :paramref:`.Operations.add_column.if_not_exists` and
:paramref:`.Operations.drop_column.if_exists` to render ``IF [NOT] EXISTS``
for ``ADD COLUMN`` and ``DROP COLUMN`` operations, a feature available on
some database backends such as PostgreSQL, MariaDB, as well as third party
backends. The parameters also support autogenerate rendering allowing them
to be turned on via a custom :class:`.Rewriter`. Pull request courtesy of
Louis-Amaury Chaib (@lachaib).
Mike Bayer [Tue, 13 May 2025 14:14:10 +0000 (10:14 -0400)]
allow pep 621 configuration
Added optional :pep:`621` support to Alembic, where a subset of the
project-centric configuration normally found in the ``alembic.ini`` file
can now be retrieved from the project-wide ``pyproject.toml`` file. A new
init template ``pyproject`` is added which illustrates a basic :pep:`621`
setup. The :pep:`621` feature supports configuration values that are
relevant to code locations and code production only; it does not
accommodate database connectivity, configuration, or logging configuration.
These latter configurational elements remain as elements that can be
present either in the ``alembic.ini`` file, or retrieved elsewhere within
the ``env.py`` file. The change also allows the ``alembic.ini`` file to
be completely optional if the ``pyproject.toml`` file contains a base
alembic configuration section.
<!-- Provide a general summary of your proposed changes in the Title field above -->
### Description
Rendering ExecuteSQLOp did not respect the `alembic_module_prefix` setting due
to having `op.` hard-coded in its template.
This change updates the render logic to use the same
`_alembic_autogenerate_prefix()` as it seems the rest of the ops use.
### Checklist
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [x] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Werezak [Sat, 26 Apr 2025 18:57:24 +0000 (14:57 -0400)]
Add file_path_separator to apply to all path splitting
Added new option to the ConfigParser config (typically via ``alembic.ini``)
``path_separator``. This new option supersedes the previous similar
option ``version_path_separator``. The new ``path_separator`` option
applies to the path splitting mechanism of both the ``version_locations``
option as well as the ``prepend_sys_path`` option, and in newly
rendered ``alembic.ini`` files will use the value ``os``, which means to
use the operating system path separator when splitting these string values
into a list of paths.
The new attribute applies necessary os-dependent path splitting to the
``prepend_sys_path`` option so that windows paths which contain drive
letters with colons are not inadvertently split, whereas previously
os-dependent path splitting were only available for the
``version_locations`` option.
Existing installations that do not have ``path_separator`` present
will continue to make use of ``version_path_separator`` when parsing the
``version_locations`` option, or splitting on spaces / commas if
``version_path_separator`` is also not present. ``prepend_sys_path`` will
continue to be split on spaces/commas/colons if ``path_separator`` is
not present. Under all of these fallback conditions, a deprecation
warning is now emitted encouraging to set ``path_separator``.
Pull request courtesy Mike Werezak.
This change also begins to move some of the role of interpreting
of specific config options into the Config object. This process will
continue as we look to add toml support to config.
Mike Bayer [Wed, 7 May 2025 15:34:49 +0000 (11:34 -0400)]
use pep639
The pyproject.toml configuration has been amended to use the updated
:pep:`639` configuration for license, which eliminates loud deprecation
warnings when building the package. Note this necessarily bumps
setuptools build requirement to 77.0.3.
Also bump black, flake8, see what's coming.
this change is modeled after the same change made in dogpile.cache
which has since been released.
Added new :meth:`.CommandLine.register_command` method to
:class:`.CommandLine`, intended to facilitate adding custom commands to
Alembic's command line tool with minimal code required; previously this
logic was embedded internally and was not publicly accessible. A new
recipe demonstrating this use is added. Pull request courtesy Mikhail
Bulash.
Added :paramref:`.op.drop_constraint.if_exists` parameter to
:func:`.op.drop_constraint` which will render "DROP CONSTRAINT IF EXISTS".
Pull request courtesy Aaron Griffin.
Also attempting to fix unrelated issue w/ test_stubs and python
versions which may be in a separate patch if it can't work here
Follow-up commit db12e1918556dd463847db9ffb3e761231df9b1d (which is probably sufficient for type checking) by making all annotations of server_default consistent.
The docstring of `alter_column` states: "Set to `None` to have the default removed". The type hint should accept None.
### Checklist
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [x] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
el [Fri, 28 Mar 2025 13:22:29 +0000 (09:22 -0400)]
fix AlterColumnOp modify_name ignorance
Fixed issue where the "modified_name" of :class:`.AlterColumnOp` would not
be considered when rendering op directives for autogenerate. While
autogenerate cannot detect changes in column name, this would nonetheless
impact approaches that made use of this attribute in rewriter recipes. Pull
request courtesy lenvk.
DanCardin [Tue, 4 Mar 2025 21:07:59 +0000 (16:07 -0500)]
Add TextClause to alter_column's server_default fields.
Given sqlalchemy models with server_defaults, alembic may autogenerate
`existing_server_default=sa.text("...")`, which fails typechecking on
the resultant migration because `TextClause` is not currently a valid
annotated type.
Mike Bayer [Tue, 4 Mar 2025 19:33:59 +0000 (14:33 -0500)]
fix template files include
Fixed an issue in the new :pep:`621` ``pyproject.toml`` layout that
prevented Alembic's template files from being included in the ``.whl`` file
in the distribution.
Mike Bayer [Wed, 26 Feb 2025 14:09:51 +0000 (09:09 -0500)]
dont stringify uq deferrable
Fixed autogenerate rendering bug where the ``deferrable`` element of
``UniqueConstraint``, a bool, were being stringified rather than repr'ed
when generating Python code.
Mike Bayer [Thu, 13 Feb 2025 16:53:25 +0000 (11:53 -0500)]
use pep 621
Installation has been converted to use :pep:`621`, e.g. ``pyproject.toml``.
note that the introduction of pyproject with min python version of 3.9
is also consumed by Black which now wants to format a few test files
differently, so that is included as well.
Mike Bayer [Sat, 8 Feb 2025 17:04:53 +0000 (12:04 -0500)]
drop SQLAlchemy 1.3, Python 3.8
Jenkins issues reveal that SQLAlchemy 1.3's testing provision has
unidentified deficiencies that are causing instability with the
mssql engine. As provisioning was highly refactored in 1.4, just drop
1.3 support rather than trying to figure out the exact thing that
changed.
Support for SQLAlchemy 1.3, which was EOL as of 2021, is now dropped from
Alembic as of version 1.15.0.
Support for Python 3.8 is dropped as of Alembic 1.15.0. Python 3.8 is EOL.
Try pytest 8.3 as this seems to be running fine for SQLAlchemy
Index autogenerate will now render labels for expressions
that use them. This is useful when applying operator classes
in PostgreSQL that can be keyed on the label name.
Peter Cock [Wed, 6 Nov 2024 17:07:02 +0000 (12:07 -0500)]
Avoid D103 linter warnings via script.py.mako
Added a basic docstring to the migration template files so that the
upgrade/downgrade methods pass the D103 linter check which requires a
docstring for public functions. Pull request courtesy Peter Cock.
refactor: add revision context to AutogenerateDiffsDetected so that wrappers may make other formatting of the diff
### Description
As discussed in #1597, AutogenerateDiffsDetected should hold contextual information so that command can be wrapped for another format (CI, pre-commit hook...)
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [x] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Tue, 24 Dec 2024 15:30:10 +0000 (10:30 -0500)]
check for variants (recursion branch) first in all cases
Fixed bug where autogen render of a "variant" type would fail to catch the
variants if the leading type were a dialect-specific type, rather than a
generic type.
Mike Bayer [Fri, 29 Nov 2024 16:58:53 +0000 (11:58 -0500)]
ensure rename_column works on SQLite
Modified SQLite's dialect to render "ALTER TABLE <t> RENAME COLUMN" when
:meth:`.Operations.alter_column` is used with a straight rename, supporting
SQLite's recently added column rename feature.
Mike Bayer [Mon, 25 Nov 2024 18:57:58 +0000 (13:57 -0500)]
support separation of Numeric/Float
in [1] we are considering separating Numeric and Float.
For Alembic PostgreSQL backend we need this isinstance therefore
to check for both Numeric and Float.
By keeping it to these two types, rather than targeting the
NumericCommon type being added in [1], the patch can work with
SQLAlchemy without the separation change as well.
Peter Cock [Thu, 7 Nov 2024 14:51:22 +0000 (09:51 -0500)]
Clarify what autogenerate compares
### Description
<!-- Describe your changes in detail -->
I was struggling with empty upgrade/downgrade functions because both my database and my ORM were up to date. I wrongly assumed the comparison was against the previous database scheme as per the prior revision script(s).
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [x] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Michael Bayer [Wed, 6 Nov 2024 23:43:43 +0000 (18:43 -0500)]
Fix `alembic.ini` templates to match `configparser` file format.
In the `alembic.ini` templates, I moved the inline comment about `version_path_separator` to their own lines as required by `configparser`.
### Description
In a recent project, I included the following configuration values in my `alembic.ini`. Note that the last line is the default line from the current generic `alembic.ini` template.
```
# version location specification; This defaults
# to migrations/versions. When using multiple version
# directories, initial revisions must be specified with --version-path.
# The path separator used here should be the separator specified by "version_path_separator" below.
version_locations = %(here)s/migrations/versions
# version path separator; As mentioned above, this is the character used to split
# version_locations. The default within new alembic.ini files is "os", which uses os.pathsep.
# If this key is omitted entirely, it falls back to the legacy behavior of splitting on spaces and/or commas.
# Valid values for version_path_separator are:
#
# version_path_separator = :
# version_path_separator = ;
# version_path_separator = space
version_path_separator = os # Use os.pathsep. Default configuration used for new projects.
```
When running `alembic check`, I encountered:
```
ValueError: 'os # Use os.pathsep. Default configuration used for new projects.' is not a valid value for version_path_separator; expected 'space', 'os', ':', ';'
```
It seemed that the comment in the last line was being included as part of the parsed config value, which should be `os`.
Alembic currently [uses `configparser.ConfigParser` from the standard libary](https://github.com/sqlalchemy/alembic/blob/2d60c77c81a72a78b575b96aef511e658073dec5/alembic/util/compat.py#L82-L89) to parse `alembic.ini` files. The [default `configparser` file format](https://docs.python.org/3/library/configparser.html#supported-ini-file-structure) requires that comments be on their own lines, although this can be customized. I changed the three copies of this line in Alembic's `alembic.ini` templates to remove the inline comments. In my case, this change fixed the `ValueError`.
This issue could also be fixed by changing [the default instance of `ConfigParser`](https://github.com/sqlalchemy/alembic/blob/2d60c77c81a72a78b575b96aef511e658073dec5/alembic/config.py#L202), using `inline_comment_prefixes=("#",)`. I imagine, however, that it might be better to use the default file format.
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [x] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Aaron Griffin [Wed, 6 Nov 2024 20:40:56 +0000 (15:40 -0500)]
Improve write_pyi usage
Fixes #1524
### Description
Due to some problems when running write_pyi, this includes the following changes:
1. move the sys.path.append up before alembic is imported - this is using global imports otherwise
2. use tox to run write_pyi in order to avoid globally installed dependencies
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [x] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
DanCardin [Wed, 6 Nov 2024 17:10:12 +0000 (12:10 -0500)]
fix: Correct the AutogenContext.metadata typing to include Sequence[MetaData].
### Description
The type annotation for AutogenContext.metadata is currently `Optional[MetaData]`, but `target_metadata` is `Union[MetaData, Sequence[MetaData], None]`. Seems like setting `target_metadata` to `[]` directly translates into the list that `AutogenContext` receives, and that the code is already coercing the potential single/sequence to always be a list.
My alembic plugin wasn't aware that this **could** be a list, and as such wasn't handling the possibility properly.
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [x] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Maciek Bryński [Thu, 31 Oct 2024 21:29:40 +0000 (17:29 -0400)]
Add ability to configure alembic_version table in DialectImpl
Added a new hook to the :class:`.DefaultImpl`
:meth:`.DefaultImpl.version_table_impl`. This allows third party dialects
to define the exact structure of the alembic_version table, to include use
cases where the table requires special directives and/or additional columns
so that it may function correctly on a particular backend. This is not
intended as a user-expansion hook, only a dialect implementation hook to
produce a working alembic_version table. Pull request courtesy Maciek
Bryński.
Maciek Bryński [Thu, 31 Oct 2024 16:47:35 +0000 (12:47 -0400)]
Fix mypy linting issues
### Description
Fixing mypy linting issues for new mypy version.
### Checklist
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [x] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
The `WARN` constant is an undocumented alias for `WARNING`. Whilst it’s not deprecated, it’s not mentioned at all in the documentation. This references one of `flake8-logging` plugins [rule](https://github.com/adamchainz/flake8-logging?tab=readme-ov-file#log009-warn-is-undocumented-use-warning-instead).
<!-- Provide a general summary of your proposed changes in the Title field above -->
### Description
Changed all uses of `logging.WARN` to `logging.WARNING`.
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [x] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Render `if_not_exists` option for CreateTableOp, CreateIndexOp, DropTableOp and DropIndexOp
Render ``if_exists`` and ``if_not_exists`` parameters in
:class:`.CreateTableOp`, :class:`.CreateIndexOp`, :class:`.DropTableOp` and
:class:`.DropIndexOp` in an autogenerate context. While Alembic does not
set these parameters during an autogenerate run, they can be enabled using
a custom :class:`.Rewriter` in the ``env.py`` file, where they will now be
part of the rendered Python code in revision files. Pull request courtesy
of Louis-Amaury Chaib (@lachaib).
Support if_exists and if_not_exists on create/drop table commands
Added support for :paramref:`.Operations.create_table.if_not_exists` and
:paramref:`.Operations.drop_table.if_exists`, adding similar functionality
to render IF [NOT] EXISTS for table operations in a similar way as with
indexes. Pull request courtesy Aaron Griffin.
Mike Bayer [Mon, 2 Sep 2024 15:33:26 +0000 (11:33 -0400)]
unpin setuptools
The pin for ``setuptools<69.3`` in ``pyproject.toml`` has been removed.
This pin was to prevent a sudden change to :pep:`625` in setuptools from
taking place which changes the file name of SQLAlchemy's source
distribution on pypi to be an all lower case name, and the change was
extended to all SQLAlchemy projects to prevent any further surprises.
However, the presence of this pin is now holding back environments that
otherwise want to use a newer setuptools, so we've decided to move forward
with this change, with the assumption that build environments will have
largely accommodated the setuptools change by now.
Enhance version_path_separator behaviour by adding a newline option
### Description
version_path_separator now consists a new option "newline" which allows you to specify multiple version locations across multiple lines like this:
```
version_locations =
/foo/versions
/bar/versions
/baz/versions
version_path_separator = newline
```
### Checklist
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [x] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
zhouyizhen [Wed, 5 Jun 2024 19:28:53 +0000 (15:28 -0400)]
Fix postgres detect serial in autogenerate (#1479)
<!-- Provide a general summary of your proposed changes in the Title field above --> Fixes: https://github.com/sqlalchemy/alembic/issues/1479
### Description
<!-- Describe your changes in detail -->
In https://github.com/sqlalchemy/alembic/issues/73, it tries to detact postgresql serial in autogenerate, so it won't take `nextval('seq'::regclass)` as server default for that column.
But it takes not effect for tables not in search path. This PR fixed it.
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [x] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
The constraint name in create_primary_key should be optional, but for batch operations is is required according to the type annotations
### Description
Changed the type annotation to `Optional[str]`
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [X] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
projects / thirdparty / sqlalchemy / alembic.git / log
