Django/Docker/Postgresql - psycopg2 error

Question

hope you can help me with my following issues. This is my first time using docker and docker-compose. I'm trying to "dockerize" my Django project running on Postgresql database and I'm having an issues with psycopg2 module. I'm using psycopg2-binary, as it's normaly the only one that works with my configuration. I've tried the standard psycopg2 package but it still doesn't work.

So I will begin by showing you some of my files:

relevant part of settings.py:

import os

# Build paths inside the project like this: os.path.join(BASE_DIR, ...)
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
TEMPLATES_DIRS = os.path.join(BASE_DIR, "templates")

# Quick-start development settings - unsuitable for production
# See https://docs.djangoproject.com/en/3.0/howto/deployment/checklist/

# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = os.environ.get('SECRET_KEY', 'changeme')


# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = True

ALLOWED_HOSTS = []
ALLOWED_HOSTS_ENV = os.environ.get('ALLOWED_HOSTS')
if ALLOWED_HOSTS_ENV:
    ALLOWED_HOSTS.extend(ALLOWED_HOSTS_ENV.split(','))



# Application definition

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'core.apps.CoreConfig',
    'django.contrib.staticfiles',
    'jquery',
    'ckeditor',
    'ckeditor_uploader',
    'crispy_forms',
    
]

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
]

ROOT_URLCONF = 'littledreamgardens.urls'

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [os.path.join(BASE_DIR, 'core/templates/core/')],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
                'core.views.category_list',
            ],
        },
    },
]

WSGI_APPLICATION = 'littledreamgardens.wsgi.application'


# Database
# https://docs.djangoproject.com/en/3.0/ref/settings/#databases

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': 'gardens',
        'USER': 'gardens',
        'PASSWORD': '***',
        'HOST': 'db',
        'PORT': 5432
    }
}

docker-compose.yml:

version: '3.7'

services:
  app:
    build:
      context: .
    ports:
      - "8000:8000"
    volumes:
      - ./littledreamgardens:/app
    command: sh -c "python manage.py runserver 0.0.0.0:8000"
    environment:
      - DEBUG=1
    depends_on:
      - db
  db:
    image: 
      postgres
    environment: 
      POSTGRES_NAME: gardens
      POSTGRES_USER: gardens
      POSTGRES_PASSWORD: ***

Dockerfile:

FROM python:3.8-alpine

ENV PATH="/scripts:${PATH}"

COPY ./requirements.txt /requirements.txt
RUN apk add --update --no-cache --virtual .tmp gcc libc-dev linux-headers postgresql-dev postgresql-client libffi libressl gcc python3-dev musl-dev jpeg-dev zlib-dev
RUN pip3 install -r /requirements.txt
RUN apk del .tmp

RUN mkdir /app
COPY ./littledreamgardens /app
WORKDIR /app
COPY ./scripts /scripts

RUN chmod +x /scripts/*

RUN mkdir -p /vol/web/media
RUN mkdir -p /vol/web/static
RUN adduser -D user
RUN chown -R user:user /vol
RUN chmod -R 755 /vol/web
USER user

CMD ["entrypoint.sh"]

Requirements.txt:

appdirs==1.4.4
asgiref==3.3.1
black==20.8b1
click==7.1.2
Django>=3.0.4,<4.0
psycopg2-binary>=2.8.6
django-ckeditor==6.0.0
django-crispy-forms==1.11.1
django-jquery==3.1.0
django-js-asset==1.2.2
django-tinymce==3.2.0
isort==5.7.0
mypy-extensions==0.4.3
pathspec==0.8.1
Pillow==8.0.1
pytz==2020.4
regex==2020.11.13
sqlparse==0.4.1
toml==0.10.2
typed-ast==1.4.2
typing-extensions==3.7.4.3
uWSGI>=2.0.18<2.1

Terminal:

WARNING: Found orphan containers (littledreamgardens_littlegreendreams_1, littledreamgardens_littledreamgardens_1) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
Starting littledreamgardens_db_1 ... done
Starting littledreamgardens_app_1 ... done
Attaching to littledreamgardens_db_1, littledreamgardens_app_1
db_1   | 
db_1   | PostgreSQL Database directory appears to contain a database; Skipping initialization
db_1   | 
db_1   | 2021-03-21 17:42:49.909 UTC [1] LOG:  starting PostgreSQL 13.2 (Debian 13.2-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
db_1   | 2021-03-21 17:42:49.909 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
db_1   | 2021-03-21 17:42:49.909 UTC [1] LOG:  listening on IPv6 address "::", port 5432
db_1   | 2021-03-21 17:42:49.992 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1   | 2021-03-21 17:42:50.056 UTC [26] LOG:  database system was shut down at 2021-03-21 17:42:38 UTC
db_1   | 2021-03-21 17:42:50.082 UTC [1] LOG:  database system is ready to accept connections
app_1  | Watching for file changes with StatReloader
app_1  | Exception in thread django-main-thread:
app_1  | Traceback (most recent call last):
app_1  |   File "/usr/local/lib/python3.8/site-packages/django/db/backends/postgresql/base.py", line 25, in <module>
app_1  |     import psycopg2 as Database
app_1  |   File "/usr/local/lib/python3.8/site-packages/psycopg2/__init__.py", line 51, in <module>
app_1  |     from psycopg2._psycopg import (                     # noqa
app_1  | ImportError: Error loading shared library libpq.so.5: No such file or directory (needed by /usr/local/lib/python3.8/site-packages/psycopg2/_psycopg.cpython-38-x86_64-linux-gnu.so)
app_1  | 
app_1  | During handling of the above exception, another exception occurred:
app_1  | 
app_1  | Traceback (most recent call last):
app_1  |   File "/usr/local/lib/python3.8/threading.py", line 932, in _bootstrap_inner
app_1  |     self.run()
app_1  |   File "/usr/local/lib/python3.8/threading.py", line 870, in run
app_1  |     self._target(*self._args, **self._kwargs)
app_1  |   File "/usr/local/lib/python3.8/site-packages/django/utils/autoreload.py", line 53, in wrapper
app_1  |     fn(*args, **kwargs)
app_1  |   File "/usr/local/lib/python3.8/site-packages/django/core/management/commands/runserver.py", line 110, in inner_run
app_1  |     autoreload.raise_last_exception()
app_1  |   File "/usr/local/lib/python3.8/site-packages/django/utils/autoreload.py", line 76, in raise_last_exception
app_1  |     raise _exception[1]
app_1  |   File "/usr/local/lib/python3.8/site-packages/django/core/management/__init__.py", line 357, in execute
app_1  |     autoreload.check_errors(django.setup)()
app_1  |   File "/usr/local/lib/python3.8/site-packages/django/utils/autoreload.py", line 53, in wrapper
app_1  |     fn(*args, **kwargs)
app_1  |   File "/usr/local/lib/python3.8/site-packages/django/__init__.py", line 24, in setup
app_1  |     apps.populate(settings.INSTALLED_APPS)
app_1  |   File "/usr/local/lib/python3.8/site-packages/django/apps/registry.py", line 114, in populate
app_1  |     app_config.import_models()
app_1  |   File "/usr/local/lib/python3.8/site-packages/django/apps/config.py", line 211, in import_models
app_1  |     self.models_module = import_module(models_module_name)
app_1  |   File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module
app_1  |     return _bootstrap._gcd_import(name[level:], package, level)
app_1  |   File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
app_1  |   File "<frozen importlib._bootstrap>", line 991, in _find_and_load
app_1  |   File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
app_1  |   File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
app_1  |   File "<frozen importlib._bootstrap_external>", line 783, in exec_module
app_1  |   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
app_1  |   File "/usr/local/lib/python3.8/site-packages/django/contrib/auth/models.py", line 2, in <module>
app_1  |     from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager
app_1  |   File "/usr/local/lib/python3.8/site-packages/django/contrib/auth/base_user.py", line 48, in <module>
app_1  |     class AbstractBaseUser(models.Model):
app_1  |   File "/usr/local/lib/python3.8/site-packages/django/db/models/base.py", line 122, in __new__
app_1  |     new_class.add_to_class('_meta', Options(meta, app_label))
app_1  |   File "/usr/local/lib/python3.8/site-packages/django/db/models/base.py", line 326, in add_to_class
app_1  |     value.contribute_to_class(cls, name)
app_1  |   File "/usr/local/lib/python3.8/site-packages/django/db/models/options.py", line 206, in contribute_to_class
app_1  |     self.db_table = truncate_name(self.db_table, connection.ops.max_name_length())
app_1  |   File "/usr/local/lib/python3.8/site-packages/django/db/__init__.py", line 28, in __getattr__
app_1  |     return getattr(connections[DEFAULT_DB_ALIAS], item)
app_1  |   File "/usr/local/lib/python3.8/site-packages/django/db/utils.py", line 214, in __getitem__
app_1  |     backend = load_backend(db['ENGINE'])
app_1  |   File "/usr/local/lib/python3.8/site-packages/django/db/utils.py", line 111, in load_backend
app_1  |     return import_module('%s.base' % backend_name)
app_1  |   File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module
app_1  |     return _bootstrap._gcd_import(name[level:], package, level)
app_1  |   File "/usr/local/lib/python3.8/site-packages/django/db/backends/postgresql/base.py", line 29, in <module>
app_1  |     raise ImproperlyConfigured("Error loading psycopg2 module: %s" % e)
app_1  | django.core.exceptions.ImproperlyConfigured: Error loading psycopg2 module: Error loading shared library libpq.so.5: No such file or directory (needed by /usr/local/lib/python3.8/site-packages/psycopg2/_psycopg.cpython-38-x86_64-linux-gnu.so)

Try cleaning up your Dockerfile, those extra PostgreSQL related stuff don't need to be in there. docs.docker.com/compose/django for example you can see it takes very little — stormlifter
– stormlifter, Commented Mar 21, 2021 at 19:56

periwinkle · Accepted Answer · 2021-03-21 20:58:09Z

1

This problem is similar to this question or this github issue

It is either an issue caused by bundling too many commands and having errors because of dependencies or it is an issue of package incompatibility. You are not pinning the psycopg2 version (>=). You may try pinning a specific version.

Psycopg2-binary is the what I use for local development, but with docker I usually stick with regular psycopg2

answered Mar 21, 2021 at 20:58

periwinkle

4164 silver badges10 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Alex Cornea Over a year ago

Yes you were right, it worked, now it seems to run fine. Thanks for your help!

Alex Cornea Over a year ago

I was able to create the docker image now without any errors

Collectives™ on Stack Overflow

Django/Docker/Postgresql - psycopg2 error

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related