pg_repack address bloat on table in postgresql14

pg_repack can be used as online operation to address bloat on a table in postgresql14.

Advantages of using pg_repack

• Releases storage from a table to disk/FS
• Rebuild a table - reduces I/O
• Release storage piled due to dead tuples as bloat & not cleared by Auto vacuum

Install pg_repack on server
 [root@linuxpg1 bin]# yum install pg_repack_14
Loaded plugins: langpacks, ulninfo
Resolving Dependencies
--> Running transaction check
---> Package pg_repack_14.x86_64 0:1.4.8-1.rhel7 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
===========================================================================================================================================
 Package                            Arch                         Version                                Repository                    Size
===========================================================================================================================================
Installing:
 pg_repack_14                       x86_64                       1.4.8-1.rhel7                          pgdg14                       127 k
Transaction Summary
===========================================================================================================================================
Install  1 Package
Total download size: 127 k
Installed size: 311 k
Is this ok [y/d/N]: y
Downloading packages:
pg_repack_14-1.4.8-1.rhel7.x86_64.rpm                                                                               | 127 kB  00:00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : pg_repack_14-1.4.8-1.rhel7.x86_64                                                                                       1/1
  Verifying  : pg_repack_14-1.4.8-1.rhel7.x86_64                                                                                       1/1
Installed:
  pg_repack_14.x86_64 0:1.4.8-1.rhel7
Complete!

[root@linuxpg1 bin]# pwd
/usr/pgsql-14/bin

[root@linuxpg1 bin]# ls
clusterdb   initdb             pg_config       pg_isready     pg_rewind        pg_waldump                  psql
createdb    pg_archivecleanup  pg_controldata  pg_receivewal  pg_test_fsync    postgres                    reindexdb
createuser  pg_basebackup      pg_ctl          pg_repack      pg_test_timing   postgresql-14-check-db-dir  vacuumdb
dropdb      pgbench            pg_dump         pg_resetwal    pg_upgrade       postgresql-14-setup
dropuser    pg_checksums       pg_dumpall      pg_restore     pg_verifybackup  postmaster
[root@linuxpg1 bin]#

switch to postgres user

-bash-4.2$ psql

psql (14.7)

Type "help" for help.

postgres=# set shared_preload_libraries = 'pg_repack';

ERROR:  parameter "shared_preload_libraries" cannot be changed without restarting the server

restart postgresql

[root@linuxpg1 bin]# systemctl restart postgresql-14

[root@linuxpg1 bin]# systemctl status postgresql-14

● postgresql-14.service - PostgreSQL 14 database server

   Loaded: loaded (/usr/lib/systemd/system/postgresql-14.service; enabled; vendor preset: disabled)

   Active: active (running) since Mon 2023-05-15 00:37:20 CDT; 19s ago

     Docs: https://www.postgresql.org/docs/14/static/

  Process: 7950 ExecStartPre=/usr/pgsql-14/bin/postgresql-14-check-db-dir ${PGDATA} (code=exited, status=0/SUCCESS)

 Main PID: 7959 (postmaster)

    Tasks: 8

   CGroup: /system.slice/postgresql-14.service

           ├─7959 /usr/pgsql-14/bin/postmaster -D /var/lib/pgsql/14/data/

           ├─7962 postgres: logger

           ├─7964 postgres: checkpointer

           ├─7965 postgres: background writer

           ├─7966 postgres: walwriter

           ├─7967 postgres: autovacuum launcher

           ├─7968 postgres: stats collector

           └─7969 postgres: logical replication launcher

May 15 00:37:20 linuxpg1 systemd[1]: Starting PostgreSQL 14 database server...

May 15 00:37:20 linuxpg1 postmaster[7959]: 2023-05-15 00:37:20.547 CDT [7959] LOG:  redirecting log output to logging collector process

May 15 00:37:20 linuxpg1 postmaster[7959]: 2023-05-15 00:37:20.547 CDT [7959] HINT:  Future log output will appear in directory "log".

May 15 00:37:20 linuxpg1 systemd[1]: Started PostgreSQL 14 database server.

[root@linuxpg1 bin]#

-bash-4.2$ psql

psql (14.7)

Type "help" for help.

postgres=# \l

                                  List of databases

   Name    |  Owner   | Encoding |   Collate   |    Ctype    |   Access privileges

-----------+----------+----------+-------------+-------------+-----------------------

 db1       | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | postgres=CTc/postgres+

           |          |          |             |             | app_ro=c/postgres

 postgres  | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |

 template0 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +

           |          |          |             |             | postgres=CTc/postgres

 template1 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +

           |          |          |             |             | postgres=CTc/postgres

(4 rows)

connect to postgres db

postgres=# \c postgres

create extension pack in postgres database

postgres=# create extension pg_repack;

CREATE EXTENSION

postgres=#

check location of pg_repack & add it to PATH varaible

-bash-4.2$ which pg_repack

/usr/pgsql-14/bin/pg_repack

-bash-4.2$ echo $PATH

/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/pgsql-14/bin

-bash-4.2$

Execute pg_repack

-bash-4.2$ pg_repack --dry-run -d postgres --table tab1

INFO: Dry run enabled, not executing repack

WARNING: relation "public.tab1" must have a primary key or not-null unique keys

-bash-4.2$

Add primary key

ALTER TABLE public.tab1 ADD CONSTRAINT tab1_pk PRIMARY KEY (a1);

Run pg_repack dry 

-bash-4.2$ pg_repack --dry-run -d postgres --table tab1

INFO: Dry run enabled, not executing repack

INFO: repacking table "public.tab1"

-bash-4.2$

Run pg_repack to address bloat

-bash-4.2$ pg_repack  -d postgres --table tab1

INFO: repacking table "public.tab1"

-bash-4.2$

PostgreSQL DBA helpful stuff!

PostgreSQL commands

# to get help on psql 

psql --help

# Examples of psql connection to DB using various switches/flags

psql postgres postgres -h 192.168.0.25 -p 5432 (-h flag to specify hostname)
psql -U postgres -h 192.168.0.25 -p 5432 -W  (-U flag to specify db user)

psql -U postgres -d db1 -h 192.168.0.25 -p 5432 (-d flag to specify db)

#to know current user

select current_user;

#to get current date

select current_date;

#to know user details

select * from pg_user;

#to know about databases

select * from pg_database;

#to know about tables

select * from pg_class;

#to know about roles

select * from pg_roles;

 

# to query parameters

SELECT name as "Parameter name", setting as value, short_desc FROM pg_settings WHERE name LIKE '%ssss%'; 

#to know relative file path

select pg_relation_filepath('course');

#get details of user connections

select state, count(*) from pg_stat_activity group by state;

select * from pg_stat_activity where state='idle';

SELECT pg_terminate_backend(pid);

#shortcut commands on psql

postgres=# \l -- list db

postgres=#\c dbname -- to connect to a particular db

# to know dead tuples in a table

postgres=#select relname, n_dead_tup from pg_stat_user_tables;

#to vacuum manually

postgres=# vacuum (verbose, analyze);

#to know size of tables

select relname as "tableName", pg_size_pretty(pg_table_size(pgc.oid)) as "spaceused"

from pg_class as pgc left join pg_namespace as pgns on (pgns.oid = pgc.relnamespace)

where nspname not in ('pg_catalog', 'information_schema') and nspname !='pg_toast' and relkind in ('r')

order by pg_table_size(pgc.oid) desc;

#Query to check is SSL is being used by sessions

SELECT datname as "Database name", usename as "User name", ssl, client_addr, application_name, backend_type
FROM pg_stat_ssl JOIN pg_stat_activity ON pg_stat_ssl.pid = pg_stat_activity.pid ORDER BY ssl;

#Indexes Info

postgres=# \di

postgres=# \di+

postgres=# \dis

# to get index, table its on & columns its created on

select row_number() over (order by c.relname), c.relname as index, t.relname as table, array_to_string(array_agg(a.attname), ', ') as column_names from pg_class c join pg_index i on c.oid = i.indexrelid and c.relkind='i' and c.relname not like 'pg_%' join pg_class t on t.oid= i.indrelid left join pg_attribute a on a.attrelid = t.oid and a.attnum = ANY(i.indkey) group by t.relname, c.relname order by c.relname;

#when last auto vacuum happened

select relname, last_vacuum, last_autovacuum from pg_stat_user_tables;

#to vacuum table

vacuum full tab1; -- tab1 is table name

#to know details about Archiving

postgres=# select name, setting, unit from pg_settings where name in

('wal_level', 'wal_keep_size', 'wal_keep_segments', 'wal_segment_size', 'archive_mode', 'archive_command', 'archive_timeout');

#to Check number of connections, super user connections, max connections

select * from (select count(*) used from pg_stat_activity) q1, (select setting::int res_for_super from pg_settings where name=$$superuser_reserved_connections$$) q2, (select setting::int max_conn from pg_settings where name=$$max_connections$$) q3;

#to take online backups

#can take backups as postgres user or create user with replication role

pg_basebackup -U username -D /location/wheretoplacebackups

pg_basebackup -v -U username -D /location/wheretoplacebackups

pg_basebackup -U username -D /location/wheretoplacebackups -h localhost

pg_basebackup -U username -D /location/wheretoplacebackups -Ft -z -P (-Ft - format tar file, z for compression P for progress)

#for upgrade

pg_upgrade

--old-datadir '/pg/9.x/data' --new-datadir '/pg/10.x/data'

--old-bindir '/pg/9.x/bin' --new-bindir '/pg/10.x/bin'

pg_upgrade -d oldCluster/data -D newCluster/data -b oldCluster/bin - B newCluster/bin

or you can export

$export PGDATAOLD = oldCluster/data

$export PGDATANEW = newCluster/data

$export PGBINOLD = oldCluster/bin

$export PGNEWOLD = newCluster/bin

$pg_upgrade

#to create db user

create user user1;

# to know the process tree for postgresql processes

pstree -p 7080 (where 7080 is postmaster pid)

PG Background Process

logger -- writes messages to error logs

checkpointer -- flushes dirty buffer to files (periodically)

background writer -- writes dirty buffer to files

wal writer -- writes wal buffer to wal file

autovaccum launcher -- performs vacuum operations

archiver-- for archiving

stats collector -- DBMS usage stats (session info, table usage stats)..etc.

PG Memory pools

shared memory -- db caching & transaction log caching

shared buffer -- frequently used blocks cached

wal buffer -- changes to db

postmaster process -- first to start, initialize memory, run background process, helps in recovery, creates background process for a client connection

maintenance_work_mem - for vacuum & create index, joins

temp_buffers -- temp tables area