Logging in Python: What Every Developer Should Know

Logging records meaningful application events so developers and operators can explain what happened before, during and after a failure. Python’s standard logging module lets applications emit messages with severity, context and tracebacks without scattering print() calls throughout production code. Good logging is selective: it communicates actionable facts without leaking credentials or overwhelming storage.

This guide covers logger configuration, levels, handlers, formatters, error tracebacks, modules, security and testing. It replaces ambiguous code fragments with complete Python 3 examples and distinguishes local troubleshooting from centralized production observability.

Start with the logging levels

The standard levels are DEBUG, INFO, WARNING, ERROR and CRITICAL. They indicate increasing severity, not simply different formatting. A successful routine request might deserve no log; a failed external dependency or unexpected exception needs enough context to investigate.

Choose thresholds according to the environment. A noisy debug configuration can be useful in a disposable test, but expensive or sensitive in production. The default behavior may filter messages below WARNING unless explicitly configured.

import logging
logging.basicConfig(level=logging.INFO)
logging.debug("detail hidden at INFO")
logging.info("worker initialized")
logging.warning("retrying dependency")

Use named loggers in modules

A module normally gets a logger with logging.getLogger(__name__). Libraries should not configure the application’s root logging policy, because the application or service owner needs to decide where messages go. The hierarchy of logger names enables different thresholds for different components.

Avoid creating a new logger name containing an individual user’s identifier or other high-cardinality value. Put selected identifiers in message context instead, with privacy controls.

import logging
logger = logging.getLogger(__name__)
def process(task_id):
    logger.info("processing task=%s", task_id)

Configure handlers and formatters deliberately

A handler sends accepted records to a destination such as standard error, a file or a queue. A formatter defines how fields such as timestamps, severity, logger name and message appear. Handlers and loggers can both filter events, so understand propagation before attaching several handlers.

For a small command-line application, one basicConfig() call in the program’s entry point may be enough. In an application server, use a centralized logging configuration or the host’s supported instrumentation rather than configuring handlers in every imported module.

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s %(levelname)s %(name)s: %(message)s",
)

Prefer lazy message interpolation

Pass format arguments separately to logger.info() and related methods. The logging module can defer formatting until the record actually needs to be emitted. An f-string is evaluated eagerly, even when that log level is disabled. This difference matters for expensive string construction.

Never place secrets, access tokens, passwords or unnecessary personal data in format arguments. Logs are copied into systems with different retention and access policies; a masked field is safer than logging a sensitive value with the intention to delete it later.

logger.debug("item %s has %d attributes", item_id, count)
# Avoid logging secrets or raw authorization headers.

Capture exceptions with useful context

Inside an exception handler, logger.exception() records an ERROR message and the active traceback. Outside that context, use exc_info=True only when you actually have meaningful exception information. Avoid logging the same failure repeatedly at every stack layer.

A good message states the operation and relevant safe identifiers rather than just “error happened.” Decide whether an exception should be retried, propagated or handled; writing a log is not a substitute for correct control flow.

try:
    result = int("not-a-number")
except ValueError:
    logger.exception("could not parse numeric field")

Manage files, rotation and processes

A plain file handler does not automatically implement your retention policy. For simple single-process scenarios, rotating file handlers may help bound file sizes. Containerized or hosted applications often prefer structured records on stdout or stderr for a collector to capture.

Multiple worker processes may need process-safe collection instead of writing concurrently to the same file without coordination. Test log rotation, permissions, disk space and behavior during service restart rather than assuming the handler alone solves operations.

from logging.handlers import RotatingFileHandler
handler = RotatingFileHandler(
    "app.log", maxBytes=2_000_000, backupCount=3
)
logger.addHandler(handler)

Keep structured and correlated context

Useful context can include a request ID, operation name, duration, outcome and sanitized entity ID. The fields should help connect a failure across components. Do not treat free-form human text as the only possible representation of an incident.

For larger systems, consider structured JSON logs, metrics and tracing together. Logs answer what happened at specific points; metrics reveal aggregates, while traces can show a request path. None should replace safe access control, sampling or retention limits.

Test log behavior and avoid duplication

Write tests that assert important messages and levels without depending on exact timestamps. In pytest, a captured-log fixture can help. Repeated calls to basicConfig() may do nothing once handlers already exist unless force=True is intentionally supplied in supported Python versions.

Double propagation can cause duplicate records if a child and parent both have handlers. Use a single clear configuration owner, and validate the output with the actual deployment runner.

Conclusion

Good Python logging makes failures diagnosable while respecting the privacy and cost of recorded information. Choose meaningful levels, one configuration owner, contextual but safe messages and tested handlers. Pair logging with responsible command-line inputs as described in the Python argument parsing guide, and consult the official Python logging reference for handler and formatter semantics.