Question

Connecting to managed PostgreSQL from functions using psycopg2

Posted on November 19, 2022
DigitalOcean Managed PostgreSQL Database DigitalOcean Functions Python
Asked by panu

Hi,

I just can’t wrap my head around this issue. Any time I try to connect to managed postgres from a function using psycopg2 the function times out.

2022-11-19T18:45:33.700648Z    stderr: The action did not initialize or run as expected.

The action exceeded its time limits of 15000 milliseconds.

Some facts:

Functions and databases are part of the same App
Database Trusted sources are off, it can be reached from anywhere
Function memory limit has been raised to 1024MB
Function timeout has been raised to 15000ms
It builds and deploys fine. I can import psycopg2 in code without errors.

Is there something missing? This should be the most basic usecase there is. Could you provide a good example/tutorial on this like you have with JS and MongoDB?

My project.yml:

packages:
  - name: sample
    environment:
      DATABASE_URL: <removed>
    functions:
      - name: hello
        runtime: python:default
        limits:
          timeout: 15000
          memory: 1024

My build.sh:

#!/bin/bash

set -e

virtualenv virtualenv
source virtualenv/bin/activate
pip install -r requirements.txt
deactivate

My requirements.txt:

psycopg2-binary==2.9.5

My __main__.py:

from urllib.parse import urlparse
import os
import psycopg2


def main(args):
    print("hello from hello")
    db_url = os.environ.get("DATABASE_URL")
    assert db_url is not None, "DATABASE_URL is missing!"
    print("db_url found")
    p = urlparse(db_url)
    print("parsed")
    conn = psycopg2.connect(
        dbname="name",
        user=p.username,
        password=p.password,
        port=p.port,
        host=p.hostname,
        sslmode="require",
    )
    print("connection created")
    conn.close()
    print("connection closed")
    return {"body": str(44)}

It runs fine if I remove database connection command from it. All the print statements work as expected. But if I have the connection statement there it does not even print anything. Running it locally works just fine.

from urllib.parse import urlparse
import os
import psycopg2


def main(args):
    print("hello from hello")
    db_url = os.environ.get("DATABASE_URL")
    assert db_url is not None, "DATABASE_URL is missing!"
    print("db_url found")
    p = urlparse(db_url)
    print("parsed")
    return {"body": str(44)}

Submit an answer

This textbox defaults to using Markdown to format your answer.

You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

panu • November 29, 2022

Accepted Answer

Getting back to this. I wrote a simple test function in go and it had the same issue.

package main

import (
	"context"
	"fmt"
	"os"

	"github.com/jackc/pgx/v4"
)

func Main(args map[string]interface{}) map[string]interface{} {
	fmt.Println("starting")
	conn, err := pgx.Connect(context.Background(), os.Getenv("DATABASE_URL"))
	if err != nil {
		fmt.Fprintf(os.Stderr, "Unable to connect to database: %v\n", err)
		os.Exit(1)
	}
	defer conn.Close(context.Background())

	var greeting string
	err = conn.QueryRow(context.Background(), "select 'Hello, world!'").Scan(&greeting)
	if err != nil {
		fmt.Fprintf(os.Stderr, "QueryRow failed: %v\n", err)
		os.Exit(1)
	}
	res := make(map[string]interface{})
	res["body"] = greeting
	fmt.Println(greeting)
	return res
}

But it seems that it is really a issue with Digital Ocean. After creating a database on another cloud provider and suplying it as a connection string it started to work flawlessly. So it seems that DO functions can’t interact with DO PostgresQL…

4906b5a70b9d4e6190a98a479c9900 • March 6, 2024

It seems you’re attempting to connect to the development database available for purchase through the app platform. However, please note that this database is exclusively accessible within the app platform ecosystem. Therefore, you must utilize a managed database instead.

Bobby Iliev

Site Moderator