starred/netbox
1
0
Fork 0
mirror of https://github.com/netbox-community/netbox.git synced 2026-01-11 21:10:29 +01:00
Code Issues 273 Packages Projects Releases 10 Wiki Activity

Allow Redis Connections to Use Sentinel #3207

New Issue
Closed
opened 2025-12-29 18:26:43 +01:00 by adam · 2 comments
adam commented 2025-12-29 18:26:43 +01:00
Owner
Copy Link

Originally created by @dstarner on GitHub (Jan 22, 2020).

Environment

  • Python version: 3.7.3
  • NetBox version: 2.7.2

Proposed Functionality

My proposal is to allow SENTINELS and SERVICE as optional keys to the REDIS dictionaries, like so. If these are both provided and non-empty, then the application would attempt to use the Redis Sentinel connection information. Note that the current way of configuring normal Redis would not change.

# Using the current method 
REDIS = {
    'webhooks': {
        'HOST': 'webhooks_redis',
        'PORT': 6379,
        'PASSWORD': '',
        'DATABASE': 0,
        'DEFAULT_TIMEOUT': 300,
        'SSL': False,
    },
    'caching': {
        'HOST': 'cache_redis',
        'PORT': 6379,
        'PASSWORD': '',
        'DATABASE': 0,
        'DEFAULT_TIMEOUT': 300,
        'SSL': False,
    }
}

# Using a Sentinel instance
REDIS = {
    'webhooks': {
        'SENTINELS': (
            ('sentinel.redis.example.com', 1234),  # Each is defined as (HOST, PORT) tuple
        ),
        'SERVICE': 'webhooks_redis',                   # Sentinel service to connect to
        'PASSWORD': '',
        'DATABASE': 0,
        'DEFAULT_TIMEOUT': 300,
        'SSL': False,
    },
    'caching': {
        'SENTINELS': (
            ('sentinel.redis.example.com', 1234),  # Each is defined as (HOST, PORT) tuple
        ),
        'SERVICE': 'webhooks_redis',                   # Sentinel service to connect to
        'PASSWORD': '',
        'DATABASE': 0,
        'DEFAULT_TIMEOUT': 300,
        'SSL': False,
    }
}

Use Case

It was nice that the webhooks and cache Redis connection settings were separated, but it would be ever nicer to be able to configure one (or both) to use Redis Sentinel connections for the sake of High Availability. Not only may this be preferred, but in some environments, it might be necessary due to the availability of developer environments.

Database Changes

None.

External Dependencies

None.

Originally created by @dstarner on GitHub (Jan 22, 2020). <!-- NOTE: This form is only for proposing specific new features or enhancements. If you have a general idea or question, please post to our mailing list instead of opening an issue: https://groups.google.com/forum/#!forum/netbox-discuss NOTE: Due to an excessive backlog of feature requests, we are not currently accepting any proposals which significantly extend NetBox's feature scope. Please describe the environment in which you are running NetBox. Be sure that you are running an unmodified instance of the latest stable release before submitting a bug report. --> ### Environment * Python version: 3.7.3 * NetBox version: 2.7.2 <!-- Describe in detail the new functionality you are proposing. Include any specific changes to work flows, data models, or the user interface. --> ### Proposed Functionality My proposal is to allow `SENTINELS` and `SERVICE` as optional keys to the `REDIS` dictionaries, like so. If these are both provided and non-empty, then the application would attempt to use the Redis Sentinel connection information. Note that the current way of configuring normal Redis would not change. ```python # Using the current method REDIS = { 'webhooks': { 'HOST': 'webhooks_redis', 'PORT': 6379, 'PASSWORD': '', 'DATABASE': 0, 'DEFAULT_TIMEOUT': 300, 'SSL': False, }, 'caching': { 'HOST': 'cache_redis', 'PORT': 6379, 'PASSWORD': '', 'DATABASE': 0, 'DEFAULT_TIMEOUT': 300, 'SSL': False, } } # Using a Sentinel instance REDIS = { 'webhooks': { 'SENTINELS': ( ('sentinel.redis.example.com', 1234), # Each is defined as (HOST, PORT) tuple ), 'SERVICE': 'webhooks_redis', # Sentinel service to connect to 'PASSWORD': '', 'DATABASE': 0, 'DEFAULT_TIMEOUT': 300, 'SSL': False, }, 'caching': { 'SENTINELS': ( ('sentinel.redis.example.com', 1234), # Each is defined as (HOST, PORT) tuple ), 'SERVICE': 'webhooks_redis', # Sentinel service to connect to 'PASSWORD': '', 'DATABASE': 0, 'DEFAULT_TIMEOUT': 300, 'SSL': False, } } ``` <!-- Convey an example use case for your proposed feature. Write from the perspective of a NetBox user who would benefit from the proposed functionality and describe how. ---> ### Use Case It was nice that the webhooks and cache Redis connection settings were separated, but it would be ever nicer to be able to configure one (or both) to use Redis Sentinel connections for the sake of High Availability. Not only may this be preferred, but in some environments, it might be necessary due to the availability of developer environments. <!-- Note any changes to the database schema necessary to support the new feature. For example, does the proposal require adding a new model or field? (Not all new features require database changes.) ---> ### Database Changes None. <!-- List any new dependencies on external libraries or services that this new feature would introduce. For example, does the proposal require the installation of a new Python package? (Not all new features introduce new dependencies.) --> ### External Dependencies None.
adam closed this issue 2025-12-29 18:26:43 +01:00
adam commented 2025-12-29 18:26:43 +01:00
Author
Owner
Copy Link

@kobayashi commented on GitHub (Jan 23, 2020):

Both of django-cacheops and django-rq support redis sentinel. So I would accept this FR. Let me start to figure out how to add them to settings.py, then make PR when accepted by other maintainers.

configuration.py is the same as @dstarner described and default comment out

# REDIS_SENTINEL = {
#     'webhooks': {
#         'SENTINELS': (
#             ('sentinel.redis.example.com', 1234),  # Each is defined as (HOST, PORT) tuple
#         ),
#         'MASTER': 'webhooks_redis',                   # Sentinel service to connect to
#         'PASSWORD': '',
#         'DATABASE': 0,
#         'SOCKET_TIMEOUT': 300,
#         'SSL': False,
#     },
#     'caching': {
#         'SENTINELS': (
#             ('sentinel.redis.example.com', 1234),  # Each is defined as (HOST, PORT) tuple
#         ),
#         'MASTER': 'webhooks_redis',                   # Sentinel service to connect to
#         'PASSWORD': '',
#         'DATABASE': 0,
#         'SOCKET_TIMEOUT': 300,
#         'SSL': False,
#     }
# }

settings.py

REDIS_SENTINEL = getattr(configuration, 'REDIS_SENTINEL', None)
if REDIS_SENTINEL:
    WEBHOOK_SENTINEL = REDIS_SENTINEL.get('webhooks', None)
    WEBHOOK_SENTINEL_SENTINELS = WEBHOOK_SENTINEL.get('SENTINELS', None)
    if not WEBHOOK_SENTINEL_SENTINELS:
        raise ImproperlyConfigured(
            "REDIS_SENTINEL section in configuration.py is missing SENTINELS subsection."
        )
    WEBHOOK_SENTINEL_MASTER = WEBHOOK_SENTINEL.get('MASTER', None)
    if not WEBHOOK_SENTINEL_MASTER:
        raise ImproperlyConfigured(
            "REDIS_SENTINEL section in configuration.py is missing MASTER subsection of webhooks."
        )
    WEBHOOK_SENTINEL_DB = WEBHOOK_SENTINEL.get('DATABASE', 0)
    WEBHOOK_SENTINEL_PASSWORD = WEBHOOK_SENTINEL.get('PASSWORD', None)
    WEBHOOK_SENTINEL_SOCKET_TIMEOUT = WEBHOOK_SENTINEL.get('SOCKET_TIMEOUT', None)
    WEBHOOK_SENTINEL_SSL = WEBHOOK_SENTINEL.get('SSL', False)

    RQ_QUEUES.extend({
        'SENTINELS': WEBHOOK_SENTINEL_SENTINELS,
        'MASTER_NAME': WEBHOOK_SENTINEL_MASTER,
        'DB': WEBHOOK_SENTINEL_DB,
        'SOCKET_TIMEOUT': WEBHOOK_SENTINEL_SOCKET_TIMEOUT,
        'PASSWORD': WEBHOOK_SENTINEL_PASSWORD,
        'SSL': WEBHOOK_SENTINEL_SSL,
    })
    
    CACHING_SENTINEL = REDIS_SENTINEL.get('caching', None)
    if CACHING_SENTINEL:
        # CACHEOPS_REDIS needs to be None for cacheops to enable redis sentinel
        CACHEOPS_REDIS = None
        CACHING_SENTINEL_LOCATIONS = CACHING_SENTINEL.get('SENTINELS', None)
        if not CACHEOPS_SENTINEL_LOCATIONS:
            raise ImproperlyConfigured(
                "REDIS_SENTINEL section in configuration.py is missing SENTINELS subsection of cacheops.")
        CACHING_SENTINEL_SERVICE_NAME = CACHING_SENTINEL.get('MASTER', None)
        if not CACHEOPS_SENTINEL_SERVICE_NAME:
            raise ImproperlyConfigured(
                "REDIS_SENTINEL section in configuration.py is missing MASTER subsection of cacheops.")
        CACHING_SENTINEL_SOCKET_TIMEOUT = CACHING_SENTINEL.get('SOCKET_TIMEOUT', None)
        CACHING_SENTINEL_DB = CACHING_SENTINEL.get('DATABASE', 0)
        CACHING_SENTINEL_PASSWORD = CACHING_SENTINEL.get('PASSWORD', None)
        CACHING_SENTINEL_SSL = CACHING_SENTINEL.get('SSL', False)
        CACHEOPS_SENTINEL = {
            'locations': [CACHING_SENTINEL_LOCATIONS],
            'service_name': CACHING_SENTINEL_SERVICE_NAME,
            'db': CACHING_SENTINEL_DB,
            'socket_timeout': CACHING_SENTINEL_SOCKET_TIMEOUT,
            'password': CACHING_SENTINEL_PASSWORD,
            'ssl': CACHING_SENTINEL_SSL,
        }
@kobayashi commented on GitHub (Jan 23, 2020): Both of django-cacheops and django-rq support redis sentinel. So I would accept this FR. Let me start to figure out how to add them to settings.py, then make PR when accepted by other maintainers. `configuration.py` is the same as @dstarner described and default comment out ```python # REDIS_SENTINEL = { # 'webhooks': { # 'SENTINELS': ( # ('sentinel.redis.example.com', 1234), # Each is defined as (HOST, PORT) tuple # ), # 'MASTER': 'webhooks_redis', # Sentinel service to connect to # 'PASSWORD': '', # 'DATABASE': 0, # 'SOCKET_TIMEOUT': 300, # 'SSL': False, # }, # 'caching': { # 'SENTINELS': ( # ('sentinel.redis.example.com', 1234), # Each is defined as (HOST, PORT) tuple # ), # 'MASTER': 'webhooks_redis', # Sentinel service to connect to # 'PASSWORD': '', # 'DATABASE': 0, # 'SOCKET_TIMEOUT': 300, # 'SSL': False, # } # } ``` `settings.py` ```python REDIS_SENTINEL = getattr(configuration, 'REDIS_SENTINEL', None) if REDIS_SENTINEL: WEBHOOK_SENTINEL = REDIS_SENTINEL.get('webhooks', None) WEBHOOK_SENTINEL_SENTINELS = WEBHOOK_SENTINEL.get('SENTINELS', None) if not WEBHOOK_SENTINEL_SENTINELS: raise ImproperlyConfigured( "REDIS_SENTINEL section in configuration.py is missing SENTINELS subsection." ) WEBHOOK_SENTINEL_MASTER = WEBHOOK_SENTINEL.get('MASTER', None) if not WEBHOOK_SENTINEL_MASTER: raise ImproperlyConfigured( "REDIS_SENTINEL section in configuration.py is missing MASTER subsection of webhooks." ) WEBHOOK_SENTINEL_DB = WEBHOOK_SENTINEL.get('DATABASE', 0) WEBHOOK_SENTINEL_PASSWORD = WEBHOOK_SENTINEL.get('PASSWORD', None) WEBHOOK_SENTINEL_SOCKET_TIMEOUT = WEBHOOK_SENTINEL.get('SOCKET_TIMEOUT', None) WEBHOOK_SENTINEL_SSL = WEBHOOK_SENTINEL.get('SSL', False) RQ_QUEUES.extend({ 'SENTINELS': WEBHOOK_SENTINEL_SENTINELS, 'MASTER_NAME': WEBHOOK_SENTINEL_MASTER, 'DB': WEBHOOK_SENTINEL_DB, 'SOCKET_TIMEOUT': WEBHOOK_SENTINEL_SOCKET_TIMEOUT, 'PASSWORD': WEBHOOK_SENTINEL_PASSWORD, 'SSL': WEBHOOK_SENTINEL_SSL, }) CACHING_SENTINEL = REDIS_SENTINEL.get('caching', None) if CACHING_SENTINEL: # CACHEOPS_REDIS needs to be None for cacheops to enable redis sentinel CACHEOPS_REDIS = None CACHING_SENTINEL_LOCATIONS = CACHING_SENTINEL.get('SENTINELS', None) if not CACHEOPS_SENTINEL_LOCATIONS: raise ImproperlyConfigured( "REDIS_SENTINEL section in configuration.py is missing SENTINELS subsection of cacheops.") CACHING_SENTINEL_SERVICE_NAME = CACHING_SENTINEL.get('MASTER', None) if not CACHEOPS_SENTINEL_SERVICE_NAME: raise ImproperlyConfigured( "REDIS_SENTINEL section in configuration.py is missing MASTER subsection of cacheops.") CACHING_SENTINEL_SOCKET_TIMEOUT = CACHING_SENTINEL.get('SOCKET_TIMEOUT', None) CACHING_SENTINEL_DB = CACHING_SENTINEL.get('DATABASE', 0) CACHING_SENTINEL_PASSWORD = CACHING_SENTINEL.get('PASSWORD', None) CACHING_SENTINEL_SSL = CACHING_SENTINEL.get('SSL', False) CACHEOPS_SENTINEL = { 'locations': [CACHING_SENTINEL_LOCATIONS], 'service_name': CACHING_SENTINEL_SERVICE_NAME, 'db': CACHING_SENTINEL_DB, 'socket_timeout': CACHING_SENTINEL_SOCKET_TIMEOUT, 'password': CACHING_SENTINEL_PASSWORD, 'ssl': CACHING_SENTINEL_SSL, } ```
adam commented 2025-12-29 18:26:44 +01:00
Author
Owner
Copy Link

@dstarner commented on GitHub (Feb 5, 2020):

@kobayashi I can probably turn my hack into the solution you have above. I already have something similar implemented, I would just need to clean it up and upstream it into the main project.

@dstarner commented on GitHub (Feb 5, 2020): @kobayashi I can probably turn my hack into the solution you have above. I already have something similar implemented, I would just need to clean it up and upstream it into the main project.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
beta
breaking change
complexity: high
complexity: low
complexity: medium
needs milestone
netbox
pending closure
plugin candidate
pull-request
Mirrored from GitHub Pull Request
 severity: high
severity: low
severity: medium
status: accepted
status: backlog
status: blocked
status: duplicate
status: needs owner
status: needs triage
status: revisions needed
status: under review
topic: GraphQL
topic: Internationalization
topic: OpenAPI
topic: UI/UX
topic: cabling
topic: event rules
topic: htmx navigation
topic: industrialization
topic: migrations
topic: plugins
topic: scripts
topic: templating
topic: testing
type: bug
type: deprecation
type: documentation
type: feature
type: housekeeping
type: translation
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#3207
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?