Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

synapse.api.errors.StoreError: 404: No row found #18056

Closed
MicaEcho opened this issue Dec 29, 2024 · 3 comments
Closed

synapse.api.errors.StoreError: 404: No row found #18056

MicaEcho opened this issue Dec 29, 2024 · 3 comments

Comments

@MicaEcho
Copy link

MicaEcho commented Dec 29, 2024 •
edited
Loading

Description

my log is being flooded by "No row found" errors. seems to come from background_process_metrics, but I don't have metrics enabled.

Steps to reproduce

I'm not sure how to reproduce it

Homeserver

another homeserver

Synapse Version

1.121.1

Installation Method

Other (please mention below)

Database

single PostgreSQL, never ported or restored, DB age is about 4 week

Workers

Multiple workers

Platform

Arch Linux, no containers
installed via pacman

Configuration

No response

Relevant log output

Dec 29 08:23:35 [redacted] synapse_federation[149953]: synapse.metrics.background_process_metrics: [process_event_queue_for_federation-1265] Background process 'process_event_queue_for_federation' threw an exception                                      
                                                         Traceback (most recent call last):                                                                                                                                                                    
                                                           File "/usr/lib/python3.13/site-packages/synapse/metrics/background_process_metrics.py", line 251, in run                                                                                            
                                                             return await func(*args, **kwargs)                                                                                                                                                                
                                                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                
                                                           File "/usr/lib/python3.13/site-packages/synapse/federation/sender/__init__.py", line 474, in _process_event_queue_loop                                                                              
                                                             last_token = await self.store.get_federation_out_pos("events")                                                                                                                                    
                                                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                    
                                                           File "/usr/lib/python3.13/site-packages/synapse/storage/databases/main/stream.py", line 1946, in get_federation_out_pos                                                                             
                                                             return await self.db_pool.simple_select_one_onecol(                                                                                                                                               
                                                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                               
                                                             ...<4 lines>...                                                                                                                                                                                   
                                                             )                                                                                                                                                                                                 
                                                             ^                                                                                                                                                                                                 
                                                           File "/usr/lib/python3.13/site-packages/synapse/storage/database.py", line 1729, in simple_select_one_onecol                                                                                        
                                                             return await self.runInteraction(                                                                                                                                                                 
                                                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                 
                                                             ...<7 lines>...                                                                                                                                                                                   
                                                             )                                                                                                                                                                                                 
                                                             ^                                                                                                                                                                                                 
                                                           File "/usr/lib/python3.13/site-packages/synapse/storage/database.py", line 952, in runInteraction                                                                                                   
                                                             return await delay_cancellation(_runInteraction())                                                                                                                                                
                                                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                
                                                           File "/usr/lib/python3.13/site-packages/twisted/internet/defer.py", line 1999, in _inlineCallbacks                                                                                                  
                                                             result = context.run(                                                                                                                                                                             
                                                                 cast(Failure, result).throwExceptionIntoGenerator, gen                                                                                                                                        
                                                             )                                                                                                                                                                                                 
                                                           File "/usr/lib/python3.13/site-packages/twisted/python/failure.py", line 519, in throwExceptionIntoGenerator                                                                                        
                                                             return g.throw(self.value.with_traceback(self.tb))                                                                                                                                                
                                                                    ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                
                                                           File "/usr/lib/python3.13/site-packages/synapse/storage/database.py", line 918, in _runInteraction
                                                             result: R = await self.runWithConnection(
                                                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                                             ...<11 lines>...
                                                             )
                                                             ^
                                                           File "/usr/lib/python3.13/site-packages/synapse/storage/database.py", line 1047, in runWithConnection
                                                             return await make_deferred_yieldable(
                                                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                                                 self._db_pool.runWithConnection(inner_func, *args, **kwargs)
                                                             )
                                                             ^
                                                           File "/usr/lib/python3.13/site-packages/twisted/python/threadpool.py", line 269, in inContext
                                                             result = inContext.theWork()  # type: ignore[attr-defined]
                                                           File "/usr/lib/python3.13/site-packages/twisted/python/threadpool.py", line 285, in <lambda>
                                                             inContext.theWork = lambda: context.call(  # type: ignore[attr-defined]
                                                                                         ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                                                 ctx, func, *args, **kw
                                                                 ^^^^^^^^^^^^^^^^^^^^^^
                                                             )
                                                             ^
                                                           File "/usr/lib/python3.13/site-packages/twisted/python/context.py", line 117, in callWithContext
                                                             return self.currentContext().callWithContext(ctx, func, *args, **kw)
                                                                    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
                                                           File "/usr/lib/python3.13/site-packages/twisted/python/context.py", line 82, in callWithContext
                                                             return func(*args, **kw)
                                                           File "/usr/lib/python3.13/site-packages/twisted/enterprise/adbapi.py", line 282, in _runWithConnection
                                                             result = func(conn, *args, **kw)
                                                           File "/usr/lib/python3.13/site-packages/synapse/storage/database.py", line 1040, in inner_func
                                                             return func(db_conn, *args, **kwargs)
                                                           File "/usr/lib/python3.13/site-packages/synapse/storage/database.py", line 780, in new_transaction
                                                             r = func(cursor, *args, **kwargs)
                                                           File "/usr/lib/python3.13/site-packages/synapse/storage/database.py", line 1780, in simple_select_one_onecol_txn
                                                             raise StoreError(404, "No row found")
                                                         synapse.api.errors.StoreError: 404: No row found
Dec 29 08:23:36 [redacted] synapse_federation[149953]: synapse.metrics.background_process_metrics: [process_event_queue_for_federation-1266] Background process 'process_event_queue_for_federation' threw an exception
                                                         Traceback (most recent call last):
                                                           File "/usr/lib/python3.13/site-packages/synapse/metrics/background_process_metrics.py", line 251, in run
                                                             return await func(*args, **kwargs)
                                                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                                           File "/usr/lib/python3.13/site-packages/synapse/federation/sender/__init__.py", line 474, in _process_event_queue_loop
                                                             last_token = await self.store.get_federation_out_pos("events")
                                                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                                           File "/usr/lib/python3.13/site-packages/synapse/storage/databases/main/stream.py", line 1946, in get_federation_out_pos
                                                             return await self.db_pool.simple_select_one_onecol(
                                                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                                             ...<4 lines>...
                                                             )
                                                             ^
                                                           File "/usr/lib/python3.13/site-packages/synapse/storage/database.py", line 952, in runInteraction                                                                                                   
                                                             return await delay_cancellation(_runInteraction())                                                                                                                                                
                                                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                                           File "/usr/lib/python3.13/site-packages/twisted/internet/defer.py", line 1999, in _inlineCallbacks
                                                             result = context.run(
                                                                 cast(Failure, result).throwExceptionIntoGenerator, gen
                                                             )
                                                           File "/usr/lib/python3.13/site-packages/twisted/python/failure.py", line 519, in throwExceptionIntoGenerator
                                                             return g.throw(self.value.with_traceback(self.tb))
                                                                    ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                                           File "/usr/lib/python3.13/site-packages/synapse/storage/database.py", line 918, in _runInteraction
                                                             result: R = await self.runWithConnection(
                                                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                                             ...<11 lines>...
                                                             )
                                                             ^
                                                           File "/usr/lib/python3.13/site-packages/synapse/storage/database.py", line 1047, in runWithConnection
                                                             return await make_deferred_yieldable(
                                                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                                                 self._db_pool.runWithConnection(inner_func, *args, **kwargs)
                                                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                                             )
                                                             ^
                                                           File "/usr/lib/python3.13/site-packages/twisted/python/threadpool.py", line 269, in inContext
                                                             result = inContext.theWork()  # type: ignore[attr-defined]
                                                           File "/usr/lib/python3.13/site-packages/twisted/python/threadpool.py", line 285, in <lambda>
                                                             inContext.theWork = lambda: context.call(  # type: ignore[attr-defined]
                                                                                         ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                                                 ctx, func, *args, **kw
                                                                 ^^^^^^^^^^^^^^^^^^^^^^
                                                             )
                                                             ^
                                                           File "/usr/lib/python3.13/site-packages/twisted/python/context.py", line 117, in callWithContext
                                                             return self.currentContext().callWithContext(ctx, func, *args, **kw)
                                                                    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
                                                           File "/usr/lib/python3.13/site-packages/twisted/python/context.py", line 82, in callWithContext
                                                             return func(*args, **kw)
                                                           File "/usr/lib/python3.13/site-packages/twisted/enterprise/adbapi.py", line 282, in _runWithConnection
                                                             result = func(conn, *args, **kw)
                                                           File "/usr/lib/python3.13/site-packages/synapse/storage/database.py", line 1040, in inner_func
                                                             return func(db_conn, *args, **kwargs)
                                                           File "/usr/lib/python3.13/site-packages/synapse/storage/database.py", line 780, in new_transaction
                                                             r = func(cursor, *args, **kwargs)
                                                           File "/usr/lib/python3.13/site-packages/synapse/storage/database.py", line 1780, in simple_select_one_onecol_txn
                                                             raise StoreError(404, "No row found")
                                                         synapse.api.errors.StoreError: 404: No row found

Anything else that would be useful to know?

No response
@lamoboos223
Copy link
Contributor

lamoboos223 commented Dec 29, 2024 •
edited
Loading

can you share your homeserver.yaml content? because I think it's generated from federation

@MicaEcho

@MicaEcho
Copy link
Author

can you share your homeserver.yaml content? because I think it's generated from federation

@MicaEcho

server_name: "[redacted for privacy]"
pid_file: /tmp/homeserver.pid
worker_app: synapse.app.homeserver
listeners:
  - port: [redacted for privacy]
    tls: false
    type: http
    x_forwarded: true
    bind_addresses: ['::1', '0.0.0.0']
    resources:
      - names: [client, federation]
        compress: false
  - port: [redacted for privacy]
    bind_address: '127.0.0.1'
    type: http
    resources:
      - names: [replication]
instance_map:
  main:
    host: localhost
    port: [redacted for privacy]
#  main_tcp:
#    host: localhost
#    port: [redacted for privacy]
  worker1:
    host: localhost
    port: [redacted for privacy]
  federation_reader1:
    host: localhost
    port: [redacted for privacy]
  federation_reader2:
    host: localhost
    port: [redacted for privacy]
  federation_reader3:
    host: localhost
    port: [redacted for privacy]
  federation_sender1:
    host: localhost
    port: [redacted for privacy]
  federation_sender2:
    host: localhost
    port: [redacted for privacy]
  federation_sender3:
    host: localhost
    port: [redacted for privacy]
  federation_sender4:
    host: localhost
    port: [redacted for privacy]
  federation_sender5:
    host: localhost
    port: [redacted for privacy]
  federation_sender6:
    host: localhost
    port: [redacted for privacy]
stream_writers:
  events: worker1
  typing: worker1
send_federation: false
outbound_federation_restricted_to:
  - federation_sender1
  - federation_sender2
  - federation_sender3
  - federation_sender4
  - federation_sender5
  - federation_sender6
run_background_tasks_on: worker1
update_user_directory_from_worker: worker1
media_instance_running_background_jobs: worker1
worker_replication_secret: "[redacted for privacy]"
redis:
  enabled: true
  path: /run/redis/redis.sock
  password: "[redacted for privacy]"
database:
  name: psycopg2
  args:
    user: [redacted for privacy]
    password: "[redacted for privacy]"
    dbname: [redacted for privacy]
    host: 127.0.0.1
    cp_min: 5
    cp_max: 10
log_config: "/etc/synapse/[redacted for privacy]"
media_store_path: [redacted for privacy]
registration_shared_secret: "[redacted for privacy]"
report_stats: false
macaroon_secret_key: "[redacted for privacy]"
form_secret: "[redacted for privacy]"
signing_key_path: "/etc/synapse/[redacted for privacy]"
trusted_key_servers:
  - server_name: "[redacted for privacy]"
admin_contact: "[redacted for privacy]"
enable_registration: true
enable_registration_without_verification: true
registration_requires_token: true
no_tls: true
max_avatar_size: 5M
allowed_avatar_mimetypes: ["image/png", "image/jpeg"]
max_pending_media_uploads: 30
max_upload_size: 512M
max_image_pixels: 40M
require_auth_for_profile_requests: true
allow_public_rooms_without_auth: true
allow_public_rooms_over_federation: true
limit_remote_rooms:
  enabled: true
  complexity: 80.0
  complexity_error: "This room is cursed! I can't let you do that."
  admins_can_join: false
require_membership_for_aliases: true
event_cache_size: 15K
caches:
  global_factor: 1.0
  per_cache_factors:
    get_users_who_share_room_with_user: 2.0
  sync_response_cache_duration: 2m
  cache_autotuning:
    max_cache_memory_usage: 8056M
    target_cache_memory_usage: 6000M
    min_cache_ttl: 5m
rc_message:
  per_second: 1
  burst_count: 20
rc_login:
  address:
    per_second: 0.15
    burst_count: 5
  account:
    per_second: 0.18
    burst_count: 5
  failed_attempts:
    per_second: 0.19
    burst_count: 7
rc_admin_redaction:
  per_second: 2
  burst_count: 50
rc_joins:
  local:
    per_second: 1
    burst_count: 20
  remote:
    per_second: 0.5
    burst_count: 15
rc_joins_per_room:
  per_second: 1
  burst_count: 10
rc_3pid_validation:
  per_second: 0.05
  burst_count: 5
rc_invites:
  per_room:
    per_second: 0.5
    burst_count: 5
  per_user:
    per_second: 0.05
    burst_count: 3
  per_issuer:
    per_second: 0.5
    burst_count: 5
rc_media_create:
  per_second: 20
  burst_count: 70
rc_federation:
  window_size: 1500
  sleep_limit: 30
  sleep_delay: 400
  reject_limit: 100
  concurrent: 10
federation_rr_transactions_per_room_per_second: 40
encryption_enabled_by_default_for_room_type: all
forget_rooms_on_leave: true
federation:
  client_timeout: 120s
  max_short_retry_delay: 10s
  max_long_retry_delay: 60s
  max_short_retries: 5
  max_long_retries: 20
  destination_min_retry_interval: 5m
  destination_retry_multiplier: 5
  destination_max_retry_interval: 5d
app_service_config_files:
  - [redacted for privacy]
  - [redacted for privacy]
  - [redacted for privacy]
url_preview_enabled: true
url_preview_ip_range_blacklist:
  - '127.0.0.0/8'
  - '10.0.0.0/8'
  - '172.16.0.0/12'
  - '192.168.0.0/16'
  - '100.64.0.0/10'
  - '192.0.0.0/24'
  - '169.254.0.0/16'
  - '192.88.99.0/24'
  - '198.18.0.0/15'
  - '192.0.2.0/24'
  - '198.51.100.0/24'
  - '203.0.113.0/24'
  - '224.0.0.0/4'
  - '::1/128'
  - 'fe80::/10'
  - 'fc00::/7'
  - '2001:db8::/32'
  - 'ff00::/8'
  - 'fec0::/10'
user_directory:
  enable: false
  search_all_users: false
  prefer_local_users: true
  show_locked_users: false
stats:
  enabled: true
enable_room_list_search: true
retention:
  enabled: true
  default_policy:
    min_lifetime: 1y
    max_lifetime: 5y
  allowed_lifetime_min: 7d
  allowed_lifetime_max: 5y
  purge_jobs:
    - interval: 7d
# vim:ft=yaml

@MicaEcho
Copy link
Author

MicaEcho commented Jan 4, 2025

Well, it stopped happening, I didn't change anything.
I'm going to close this.

@MicaEcho MicaEcho closed this as completed Jan 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@lamoboos223 @MicaEcho