Tom Lane [Sun, 21 May 2006 20:20:05 +0000 (20:20 +0000)]
Modify libpq's string-escaping routines to be aware of encoding considerations
and standard_conforming_strings. The encoding changes are needed for proper
escaping in multibyte encodings, as per the SQL-injection vulnerabilities
noted in CVE-2006-2313 and CVE-2006-2314. Concurrent fixes are being applied
to the server to ensure that it rejects queries that may have been corrupted
by attempted SQL injection, but this merely guarantees that unpatched clients
will fail rather than allow injection. An actual fix requires changing the
client-side code. While at it we have also fixed these routines to understand
about standard_conforming_strings, so that the upcoming changeover to SQL-spec
string syntax can be somewhat transparent to client code.
Since the existing API of PQescapeString and PQescapeBytea provides no way to
inform them which settings are in use, these functions are now deprecated in
favor of new functions PQescapeStringConn and PQescapeByteaConn. The new
functions take the PGconn to which the string will be sent as an additional
parameter, and look inside the connection structure to determine what to do.
So as to provide some functionality for clients using the old functions,
libpq stores the latest encoding and standard_conforming_strings values
received from the backend in static variables, and the old functions consult
these variables. This will work reliably in clients using only one Postgres
connection at a time, or even multiple connections if they all use the same
encoding and string syntax settings; which should cover many practical
scenarios.
Clients that use homebrew escaping methods, such as PHP's addslashes()
function or even hardwired regexp substitution, will require extra effort
to fix :-(. It is strongly recommended that such code be replaced by use of
PQescapeStringConn/PQescapeByteaConn if at all feasible.
Tom Lane [Sun, 21 May 2006 20:11:25 +0000 (20:11 +0000)]
Add a new GUC parameter backslash_quote, which determines whether the SQL
parser will allow "\'" to be used to represent a literal quote mark. The
"\'" representation has been deprecated for some time in favor of the
SQL-standard representation "''" (two single quote marks), but it has been
used often enough that just disallowing it immediately won't do. Hence
backslash_quote allows the settings "on", "off", and "safe_encoding",
the last meaning to allow "\'" only if client_encoding is a valid server
encoding. That is now the default, and the reason is that in encodings
such as SJIS that allow 0x5c (ASCII backslash) to be the last byte of a
multibyte character, accepting "\'" allows SQL-injection attacks as per
CVE-2006-2314 (further details will be published after release). The
"on" setting is available for backward compatibility, but it must not be
used with clients that are exposed to untrusted input.
Thanks to Akio Ishida and Yasuo Ohgaki for identifying this security issue.
Tom Lane [Sun, 21 May 2006 20:06:18 +0000 (20:06 +0000)]
Change the backend to reject strings containing invalidly-encoded multibyte
characters in all cases. Formerly we mostly just threw warnings for invalid
input, and failed to detect it at all if no encoding conversion was required.
The tighter check is needed to defend against SQL-injection attacks as per
CVE-2006-2313 (further details will be published after release). Embedded
zero (null) bytes will be rejected as well. The checks are applied during
input to the backend (receipt from client or COPY IN), so it no longer seems
necessary to check in textin() and related routines; any string arriving at
those functions will already have been validated. Conversion failure
reporting (for characters with no equivalent in the destination encoding)
has been cleaned up and made consistent while at it.
Also, fix a few longstanding errors in little-used encoding conversion
routines: win1251_to_iso, win866_to_iso, euc_tw_to_big5, euc_tw_to_mic,
mic_to_euc_tw were all broken to varying extents.
Patches by Tatsuo Ishii and Tom Lane. Thanks to Akio Ishida and Yasuo Ohgaki
for identifying the security issues.
Tom Lane [Fri, 19 May 2006 16:30:50 +0000 (16:30 +0000)]
Fix nasty bug in nodeIndexscan.c's detection of duplicate tuples during
a multiple (OR'ed) indexscan. It was checking for duplicate
tuple->t_data->t_ctid, when what it should be checking is tuple->t_self.
The trouble situation occurs when a live tuple has t_ctid not pointing to
itself, which can happen if an attempted UPDATE was rolled back. After a
VACUUM, an unrelated tuple could be installed where the failed update tuple
was, leading to one live tuple's t_ctid pointing to an unrelated tuple.
If one of these tuples is fetched by an earlier OR'ed indexscan and the other
by a later indexscan, nodeIndexscan.c would incorrectly ignore the second
tuple. The bug exists in all 7.4.* and 8.0.* versions, but not in earlier
or later branches because this code was only used in those releases. Per
trouble report from Rafael Martinez Guerrero.
Tom Lane [Fri, 12 May 2006 22:44:50 +0000 (22:44 +0000)]
Fix the sense of the test on DH_check()'s return value. This was preventing
custom-generated DH parameters from actually being used by the server.
Found by Michael Fuhr.
Bruce Momjian [Mon, 8 May 2006 02:18:21 +0000 (02:18 +0000)]
Build server libpgport with all non-FRONTEND object files. This is to
fix a Win32 bug where pipe.c included a file that used FRONTEND, but it
wasn't on the server-build list.
Tom Lane [Wed, 19 Apr 2006 16:15:41 +0000 (16:15 +0000)]
Fix ancient memory leak in PQprintTuples(); our code no longer uses this
routine, but perhaps some applications do. Found by Martijn van Oosterhout
using Coverity.
Tom Lane [Thu, 13 Apr 2006 18:01:45 +0000 (18:01 +0000)]
Fix similar_escape() so that SIMILAR TO works properly for patterns involving
alternatives ("|" symbol). The original coding allowed the added ^ and $
constraints to be absorbed into the first and last alternatives, producing
a pattern that would match more than it should. Per report from Eric Noriega.
I also changed the pattern to add an ARE director ("***:"), ensuring that
SIMILAR TO patterns do not change behavior if regex_flavor is changed. This
is necessary to make the non-capturing parentheses work, and seems like a
good idea on general principles.
Back-patched as far as 7.4. 7.3 also has the bug, but a fix seems impractical
because that version's regex engine doesn't have non-capturing parens.
Tom Lane [Wed, 29 Mar 2006 15:16:00 +0000 (15:16 +0000)]
TablespaceCreateDbspace should function normally even on platforms that do not
have symlinks (ie, Windows). Although it'll never be called on to do anything
useful during normal operation on such a platform, it's still needed to
re-create dropped directories during WAL replay.
Tom Lane [Tue, 28 Mar 2006 21:17:44 +0000 (21:17 +0000)]
Repair longstanding error in btree xlog replay: XLogReadBuffer should be
passed extend = true whenever we are reading a page we intend to reinitialize
completely, even if we think the page "should exist". This is because it
might indeed not exist, if the relation got truncated sometime after the
current xlog record was made and before the crash we're trying to recover
from. These two thinkos appear to explain both of the old bug reports
discussed here:
http://archives.postgresql.org/pgsql-hackers/2005-05/msg01369.php
Neil Conway [Mon, 20 Feb 2006 20:10:43 +0000 (20:10 +0000)]
Fix three Python reference leaks in PLy_traceback(). This would result
in leaking memory when invoking a PL/Python procedure that raises an
exception. Unfortunately this still leaks memory, but at least the
largest leak has been plugged.
This patch also fixes a reference counting mistake in PLy_modify_tuple()
for 8.0, 8.1 and HEAD: we don't actually own a reference to `platt', so
we shouldn't Py_DECREF() it.
Neil Conway [Sat, 18 Feb 2006 20:48:56 +0000 (20:48 +0000)]
Patch from Marko Kreen:
pgcrypto crypt()/md5 and hmac() leak memory when compiled against
OpenSSL as openssl.c digest ->reset will do two DigestInit calls
against a context. This happened to work with OpenSSL 0.9.6
but not with 0.9.7+.
Reason for the messy code was that I tried to avoid creating
wrapper structure to transport algorithm info and tried to use
OpenSSL context for it. The fix is to create wrapper structure.
It also uses newer digest API to avoid memory allocations
on reset with newer OpenSSLs.
Tom Lane [Tue, 14 Feb 2006 17:20:17 +0000 (17:20 +0000)]
Move btbulkdelete's vacuum_delay_point() call to a place in the loop where
we are not holding a buffer content lock; where it was, InterruptHoldoffCount
is positive and so we'd not respond to cancel signals as intended. Also
add missing vacuum_delay_point() call in btvacuumcleanup. This should fix
complaint from Evgeny Gridasov about failure to respond to SIGINT/SIGTERM
in a timely fashion (bug #2257).
Tom Lane [Mon, 13 Feb 2006 16:22:38 +0000 (16:22 +0000)]
Fix qual_is_pushdown_safe to not try to push down quals involving a whole-row
Var referencing the subselect output. While this case could possibly be made
to work, it seems not worth expending effort on. Per report from Magnus
Naeslund(f).
Tom Lane [Sun, 12 Feb 2006 22:33:14 +0000 (22:33 +0000)]
Fix bug in SET SESSION AUTHORIZATION that allows unprivileged users to crash
the server, if it has been compiled with Asserts enabled (CVE-2006-0553).
Thanks to Akio Ishida for reporting this problem.
Tom Lane [Fri, 10 Feb 2006 19:01:33 +0000 (19:01 +0000)]
Change search for default operator classes so that it examines all opclasses
regardless of the current schema search path. Since CREATE OPERATOR CLASS
only allows one default opclass per datatype regardless of schemas, this
should have minimal impact, and it fixes problems with failure to find a
desired opclass while restoring dump files. Per discussion at
http://archives.postgresql.org/pgsql-hackers/2006-02/msg00284.php.
Remove now-redundant-or-unused code in typcache.c and namespace.c,
and backpatch as far as 8.0.
Tom Lane [Sun, 5 Feb 2006 20:59:06 +0000 (20:59 +0000)]
Fix pg_restore to properly discard COPY data when trying to continue
after an error in a COPY statement. Formerly it thought the COPY data
was SQL commands, and got quite confused.
Bruce Momjian [Wed, 1 Feb 2006 00:47:03 +0000 (00:47 +0000)]
Set progname early in the postmaster/postgres binary, rather than doing
it later. This fixes a problem where EXEC_BACKEND didn't have progname
set, causing a segfault if log_min_messages was set below debug2 and our
own snprintf.c was being used.
Tom Lane [Mon, 30 Jan 2006 16:19:12 +0000 (16:19 +0000)]
Fix ALTER COLUMN TYPE bug: it sometimes tried to drop UNIQUE or PRIMARY KEY
constraints before FOREIGN KEY constraints that depended on them. Originally
reported by Neil Conway on 29-Jun-2005. Patch by Nakano Yoshihisa.
Andrew Dunstan [Sat, 28 Jan 2006 16:22:49 +0000 (16:22 +0000)]
Undo perl's nasty locale setting on Windows. Since we can't do that as
elsewhere by setting the environment appropriately, we make perl do it
right after interpreter startup by calling its POSIX::setlocale().
Tom Lane [Thu, 26 Jan 2006 17:08:34 +0000 (17:08 +0000)]
Fix display of whole-row Var appearing at the top level of a SELECT list.
While we normally prefer the notation "foo.*" for a whole-row Var, that does
not work at SELECT top level, because in that context the parser will assume
that what is wanted is to expand the "*" into a list of separate target
columns, yielding behavior different from a whole-row Var. We have to emit
just "foo" instead in that context. Per report from Sokolov Yura.
Tom Lane [Sat, 21 Jan 2006 04:38:36 +0000 (04:38 +0000)]
Repair longstanding bug in slru/clog logic: it is possible for two backends
to try to create a log segment file concurrently, but the code erroneously
specified O_EXCL to open(), resulting in a needless failure. Before 7.4,
it was even a PANIC condition :-(. Correct code is actually simpler than
what we had, because we can just say O_CREAT to start with and not need a
second open() call. I believe this accounts for several recent reports of
hard-to-reproduce "could not create file ...: File exists" errors in both
pg_clog and pg_subtrans.
Tom Lane [Thu, 19 Jan 2006 20:28:57 +0000 (20:28 +0000)]
Avoid crashing if relcache flush occurs while trying to load data into an
index's support-function cache (in index_getprocinfo). Since none of that
data can change for an index that's in active use, it seems sufficient to
treat all open indexes the same way we were treating "nailed" system indexes
--- that is, just re-read the pg_class row and leave the rest of the relcache
entry strictly alone. The pg_class re-read might not be strictly necessary
either, but since the reltablespace and relfilenode can change in normal
operation it seems safest to do it. (We don't support changing any of the
other info about an index at all, at the moment.)
Back-patch as far as 8.0. It might be possible to adapt the patch to 7.4,
but it would take more work than I care to expend for such a low-probability
problem. 7.3 is out of luck for sure.
Tom Lane [Thu, 19 Jan 2006 04:45:58 +0000 (04:45 +0000)]
It turns out that TablespaceCreateDbspace fails badly if a relcache flush
occurs when it tries to heap_open pg_tablespace. When control returns to
smgrcreate, that routine will be holding a dangling pointer to a closed
SMgrRelation, resulting in mayhem. This is of course a consequence of
the violation of proper module layering inherent in having smgr.c call
a tablespace command routine, but the simplest fix seems to be to change
the locking mechanism. There's no real need for TablespaceCreateDbspace
to touch pg_tablespace at all --- it's only opening it as a way of locking
against a parallel DROP TABLESPACE command. A much better answer is to
create a special-purpose LWLock to interlock these two operations.
This drops TablespaceCreateDbspace quite a few layers down the food chain
and makes it something reasonably safe for smgr to call.
Tom Lane [Tue, 17 Jan 2006 17:33:37 +0000 (17:33 +0000)]
Repair problems with the result of lookup_rowtype_tupdesc() possibly being
discarded by cache flush while still in use. This is a minimal patch that
just copies the tupdesc anywhere it could be needed across a flush. Applied
to back branches only; Neil Conway is working on a better long-term solution
for HEAD.
Neil Conway [Sun, 15 Jan 2006 22:47:21 +0000 (22:47 +0000)]
When using GCC on AMD64 and PPC, ECPGget_variable() takes a va_list *, not
a va_list. Christof Petig's previous patch made this change, but neglected
to update ecpglib/descriptor.c, resulting in a compiler warning (and a
likely runtime crash) on AMD64 and PPC.
Tom Lane [Thu, 12 Jan 2006 21:49:19 +0000 (21:49 +0000)]
Repair "Halloween problem" in EvalPlanQual: a tuple that's been inserted by
our own command (or more generally, xmin = our xact and cmin >= current
command ID) should not be seen as good. Else we may try to update rows
we already updated. This error was inserted last August while fixing the
even bigger problem that the old coding wouldn't see *any* tuples inserted
by our own transaction as good. Per report from Euler Taveira de Oliveira.
Tom Lane [Thu, 12 Jan 2006 19:24:02 +0000 (19:24 +0000)]
Use a more bulletproof test for whether finite() and isinf() are present.
It seems that recent gcc versions can optimize away calls to these functions
even when the functions do not exist on the platform, resulting in a bogus
positive result. Avoid this by using a non-constant argument and ensuring
that the function result is not simply discarded. Per report from
François Laupretre.
Tom Lane [Tue, 10 Jan 2006 20:16:25 +0000 (20:16 +0000)]
Fix pg_tzset() to ensure that 'lclmem' (the static variable holding
the localtime timezone data) is not overwritten until we know the data
is good. tzload() is capable of failing after having begun modifying
the struct it's pointed at, and in such cases the static data was left
in a corrupt state. Bug does not exist pre-8.0 (since we didn't have
this code then) nor post-8.0 (since we already changed the code to
tzload into local variables initially). Per report from Nick Martens.
Neil Conway [Tue, 10 Jan 2006 00:33:48 +0000 (00:33 +0000)]
In PLy_function_build_args(), the code loops repeatedly, constructing
one argument at a time and then inserting the argument into a Python
list via PyList_SetItem(). This "steals" the reference to the argument:
that is, the reference to the new list member is now held by the Python
list itself. This works fine, except if an elog occurs. This causes the
function's PG_CATCH() block to be invoked, which decrements the
reference counts on both the current argument and the list of arguments.
If the elog happens to occur during the second or subsequent iteration
of the loop, the reference count on the current argument will be
decremented twice.
The fix is simple: set the local pointer to the current argument to NULL
immediately after adding it to the argument list. This ensures that the
Py_XDECREF() in the PG_CATCH() block doesn't double-decrement.
Tom Lane [Mon, 9 Jan 2006 21:16:37 +0000 (21:16 +0000)]
Fix pg_dump to add the required OPERATOR() decoration to schema-qualified
operator names. This is needed when dumping operator definitions that have
COMMUTATOR (or similar) links to operators in other schemas.
Apparently Daniel Whitter is the first person ever to try this :-(
Tom Lane [Sat, 7 Jan 2006 22:46:07 +0000 (22:46 +0000)]
Add RelationOpenSmgr() calls to ensure rd_smgr is valid when we try to
use it. While it normally has been opened earlier during btree index
build, testing shows that it's possible for the link to be closed again
if an sinval reset occurs while the index is being built.
Tom Lane [Fri, 6 Jan 2006 02:58:40 +0000 (02:58 +0000)]
Fix Windows-only postmaster code to reject a connection request and continue,
rather than elog(FATAL), when there is no more room in ShmemBackendArray.
This is a security issue since too many connection requests arriving close
together could cause the postmaster to shut down, resulting in denial of
service. Reported by Yoshiyuki Asaba, fixed by Magnus Hagander.
Tom Lane [Fri, 6 Jan 2006 00:16:09 +0000 (00:16 +0000)]
Convert Assert checking for empty page into a regular test and elog.
The consequences of overwriting a non-empty page are bad enough that
we should not omit this test in production builds.
Tom Lane [Fri, 6 Jan 2006 00:04:33 +0000 (00:04 +0000)]
Fix ReadBuffer() to correctly handle the case where it's trying to extend
the relation but it finds a pre-existing valid buffer. The buffer does not
correspond to any page known to the kernel, so we *must* do smgrextend to
ensure that the space becomes allocated. The 7.x branches all do this
correctly, but the corner case got lost somewhere during 8.0 bufmgr rewrites.
(My fault no doubt :-( ... I think I assumed that such a buffer must be
not-BM_VALID, which is not so.)
Joe Conway [Tue, 3 Jan 2006 23:47:24 +0000 (23:47 +0000)]
When the remote query result has a different number of columns
than the local query specifies (e.g. in the FROM clause),
throw an ERROR (instead of crashing). Fix for bug #2129 reported
by Akio Iwaasa.
Tom Lane [Tue, 3 Jan 2006 23:46:44 +0000 (23:46 +0000)]
There is a signedness bug in Openwall gen_salt code that pgcrypto uses.
This makes the salt space for md5 and xdes algorithms a lot smaller than
it should be.
Tom Lane [Tue, 3 Jan 2006 22:48:28 +0000 (22:48 +0000)]
Add checks to verify that a plpgsql function returning a rowtype is actually
returning the rowtype it's supposed to return. Per reports from David Niblett
and Michael Fuhr.
Neil Conway [Sun, 1 Jan 2006 10:14:59 +0000 (10:14 +0000)]
Remove DOS line endings ("\r\n") from several .po files. DOS line endings
are inconsistent with the rest of the .po files, and apparently cause
problems for Sun's cc. Per report on IRC from "bitvector2".
Neil Conway [Thu, 29 Dec 2005 21:47:49 +0000 (21:47 +0000)]
Index: src/pl/plpython/plpython.c
===================================================================
RCS file: /Users/neilc/postgres/cvs_root/pgsql/src/pl/plpython/plpython.c,v
retrieving revision 1.67
diff -c -r1.67 plpython.c
*** src/pl/plpython/plpython.c 26 Dec 2005 04:28:48 -0000 1.67
--- src/pl/plpython/plpython.c 29 Dec 2005 16:54:57 -0000
***************
*** 2,8 ****
* plpython.c - python as a procedural language for PostgreSQL
*
* This software is copyright by Andrew Bosma
! * but is really shameless cribbed from pltcl.c by Jan Weick, and
* plperl.c by Mark Hollomon.
*
* The author hereby grants permission to use, copy, modify,
--- 2,8 ----
* plpython.c - python as a procedural language for PostgreSQL
*
* This software is copyright by Andrew Bosma
! * but is really shamelessly cribbed from pltcl.c by Jan Wieck, and
* plperl.c by Mark Hollomon.
*
* The author hereby grants permission to use, copy, modify,
***************
*** 1996,2002 ****
int i,
rv;
PLyPlanObject *plan;
- char *nulls;
MemoryContext oldcontext;
Tatsuo Ishii [Sat, 24 Dec 2005 10:11:32 +0000 (10:11 +0000)]
Fix long standing Asian multibyte charsets bug.
See:
Subject: [HACKERS] bugs with certain Asian multibyte charsets
From: Tatsuo Ishii <ishii@sraoss.co.jp>
To: pgsql-hackers@postgresql.org
Date: Sat, 24 Dec 2005 18:25:33 +0900 (JST)
Tom Lane [Fri, 23 Dec 2005 22:34:33 +0000 (22:34 +0000)]
Fix make_relative_path() to support cases where target_path and bin_path
differ by more than the last directory component. Instead of insisting
that they match up to the last component, accept whatever common prefix
they have, and try to replace the non-matching part of bin_path with
the non-matching part of target_path in the actual executable's path.
In one way this is tighter than the old code, because it insists on
a match to the part of bin_path we want to substitute for, rather than
blindly stripping one directory component from the executable's path.
Per gripe from Martin Pitt and subsequent discussion.
Tom Lane [Thu, 22 Dec 2005 22:50:14 +0000 (22:50 +0000)]
Adjust string comparison so that only bitwise-equal strings are considered
equal: if strcoll claims two strings are equal, check it with strcmp, and
sort according to strcmp if not identical. This fixes inconsistent
behavior under glibc's hu_HU locale, and probably under some other locales
as well. Also, take advantage of the now-well-defined behavior to speed up
texteq, textne, bpchareq, bpcharne: they may as well just do a bitwise
comparison and not bother with strcoll at all.
NOTE: affected databases may need to REINDEX indexes on text columns to be
sure they are self-consistent.
Tom Lane [Wed, 14 Dec 2005 17:06:51 +0000 (17:06 +0000)]
Defend against crash while processing Describe Statement or Describe Portal
messages, when client attempts to execute these outside a transaction (start
one) or in a failed transaction (reject message, except for COMMIT/ROLLBACK
statements which we can handle). Per report from Francisco Figueiredo Jr.
Tom Lane [Thu, 8 Dec 2005 19:19:45 +0000 (19:19 +0000)]
Fix bgwriter's failure to release buffer pins and open files after an
error. This probably explains bug #2099 and could also account for
mysterious VACUUM hangups.
Tom Lane [Tue, 6 Dec 2005 16:50:55 +0000 (16:50 +0000)]
In a nestloop inner indexscan, it's OK to use pushed-down baserestrictinfo
clauses even if it's an outer join. This is a corner case since such
clauses could only arise from weird OUTER JOIN ON conditions, but worth
fixing. Per example from Ron at cheapcomplexdevices.com.
Tom Lane [Thu, 1 Dec 2005 20:24:49 +0000 (20:24 +0000)]
Retry in FileRead and FileWrite if Windows returns ERROR_NO_SYSTEM_RESOURCES.
Also add a retry for Unixen returning EINTR, which hasn't been reported
as an issue but at least theoretically could be. Patch by Qingqing Zhou,
some minor adjustments by me.