feike
Joined 8 years ago
Joined 8 years ago
The times I've tested it, the improvement is very small as compared to plain copy, or copy with CSV, whereas it does require more work and thought upfront to ensure the binary actually works correctly.
(On some metrics data internally, I have 98% reduction in size of the data).
One of the reasons this works is due to only having to pay the per-tuple overhead once per grouped row, which could be as much as a 1000 rows.
The other is the compression algorithm, which can be TimescaleDB or plain PostgreSQL TOAST
https://www.timescale.com/blog/time-series-compression-algor... https://www.postgresql.org/docs/current/storage-toast.html
INSERT INTO tbl1 VALUES ($1, $2) \bind 'first value' 'second value' \g
https://www.postgresql.org/docs/current/app-psql.html#APP-PS...For older versions, you can do:
\set v_x 'first value'
\set v_y 'second value'
INSERT INTO tbl1 VALUES (:'v_x', :'v_y');
\set v_x 'next value'
INSERT INTO tbl1 VALUES (:'v_x', :'v_y');The oldest I can find is from 1998 (PostgreSQL 6.3), but it was probably in use even before.
> Postgres offers substantial additional power by incorporating the following four additional basic concepts in such a way that users can easily extend the system:
classes inheritance types functions
Other features provide additional power and flexibility:
constraints triggers rules transaction integrity
These features put Postgres into the category of databases referred to as object-relational
> The "trusted" version of PL/Rust uses a unique fork of Rust's std entitled postgrestd when compiling LANGUAGE plrust user functions.
08:25:37.114 UTC [1456] LOG: statement: CREATE DATABASE "test_1675239937111796557" WITH template = test_template
[noise]
08:25:48.002 UTC [1486] LOG: statement: DROP DATABASE "test_1675239947937354435"
Start time of first test:
2023-02-01 08:25:03.633 UTCFinish time of last test: 2023-02-01 08:26:13.861 UTC
82 tests, or 0.856 seconds per test (sequentially).
In parallel, we take 6.941 seconds for 82 tests, or 0.085 seconds per test.
- we create a template database using the migrations
- for *every* integration test we do `CREATE DATABASE test123 TEMPLATE test_template;`
- we tune the PostgreSQL instance inside Docker to speed up things, for exampling disabling synchronous_commit
On a successful test, we drop the test123 database. On a failed test, we keep the database around, so we can inspect it a bit.The really great thing about this approach (IMHO), is that you can validate certain constraint violations.
For example, exclusion constraints are great for modelling certain use cases where overlapping ranges should be avoided. In our (go) code, the test cases can use the sqlstate code, or the constraint name to figure out if we hit the error we expect to hit.
This approach is pretty much as fast as our unit tests (your mileage may vary), but it prevents way more bugs from being merged into our codebase.
"and will support PostgreSQL 13 and all the future releases of PostgreSQL."
You could compose a SQL query that allows you to map multiple resultsets to 1 resultset, although that feels a bit awkward.
WITH a AS (
insert into a (k, v) values ('a', 1.0) returning *
), b AS (
insert into b (k, v) values ('b', 2.0) returning *
)
SELECT
row_to_json(a)
FROM
a
UNION ALL
SELECT
row_to_json(b)
FROM
b;
Returns: row_to_json
--------------------------
{"a_id":1,"k":"a","v":1}
{"b_id":1,"k":"b","v":2}
(2 rows)His site is a great resource for anyone wanting to take a deeper dive on SQL performance:
https://use-the-index-luke.com/sql/partial-results/fetch-nex...
SELECT
lineno::int AS line,
((lineno-3)/6)::smallint AS card,
((lineno-3)%6)::smallint AS y,
(col - 1)::smallint AS x,
value::smallint AS value
FROM
regexp_split_to_table($1, '\n') WITH ORDINALITY AS sub(line, lineno)
CROSS JOIN
regexp_split_to_table(ltrim(line, ' '), '(\s+|,)') WITH ORDINALITY AS sub2(value, col)
WHERE
line != ''
AND value != ''
I am therefore quite sad to see this happen. It won't be easy to get feature parity with this great product.
I sincerely hope this is a reversible decision, or perhaps the postgres project could even absorb it into contrib.