Oasis gRPC (grpc.oasis.io) and Emerald Web3 Gateway (emerald.oasis.io) are down
Resolved
Nov 7, 2024 at 10:02pm UTC
Requests to grpc.oasis.io, sapphire.oasis.io, and emerald.oasis.io were intermittently timing out due to load balancer misconfigurations and Oasis node-specific issues.
To resolve the issues we have reconfigured load balancer pools to include more other production nodes and removed problematic nodes. We have also restarted affected components (e.g., proxies) and updated the Linux kernel to address filesystem inconsistencies on impacted nodes and restore stability.
Services are now operational, and traffic is routing correctly. In the future, we plan to stabilize nodes by removing some unneeded components and enhancing monitoring to detect these kinds of issues.
Affected services
Updated
Nov 7, 2024 at 9:21pm UTC
Oasis gRPC (grpc.oasis.io) and Emerald Web3 Gateway (emerald.oasis.io) recovered.
Affected services
Updated
Nov 7, 2024 at 8:59pm UTC
Oasis gRPC (grpc.oasis.io) and Emerald Web3 Gateway (emerald.oasis.io) went down.
Affected services
Updated
Nov 7, 2024 at 12:01pm UTC
Oasis gRPC (grpc.oasis.io) recovered.
Affected services
Updated
Nov 7, 2024 at 12:00pm UTC
Oasis gRPC (grpc.oasis.io) went down.
Affected services
Updated
Nov 7, 2024 at 6:13am UTC
Oasis gRPC (grpc.oasis.io) recovered.
Affected services
Updated
Nov 7, 2024 at 6:08am UTC
Oasis gRPC (grpc.oasis.io) went down.
Affected services
Updated
Nov 7, 2024 at 5:59am UTC
Oasis gRPC (grpc.oasis.io) recovered.
Affected services
Updated
Nov 7, 2024 at 5:47am UTC
Oasis gRPC (grpc.oasis.io) went down.
Affected services
Updated
Nov 7, 2024 at 5:13am UTC
Emerald Web3 Gateway (emerald.oasis.io) recovered.
Affected services
Updated
Nov 7, 2024 at 4:40am UTC
Emerald Web3 Gateway (emerald.oasis.io) went down.
Affected services
Updated
Nov 7, 2024 at 4:22am UTC
Oasis gRPC (grpc.oasis.io) recovered.
Affected services
Updated
Nov 7, 2024 at 3:45am UTC
Oasis gRPC (grpc.oasis.io) went down.
Affected services
Updated
Nov 7, 2024 at 3:24am UTC
Emerald Web3 Gateway (emerald.oasis.io) recovered.
Affected services
Updated
Nov 7, 2024 at 3:18am UTC
Oasis gRPC (grpc.oasis.io) recovered.
Affected services
Updated
Nov 7, 2024 at 3:14am UTC
Oasis gRPC (grpc.oasis.io) went down.
Affected services
Updated
Nov 7, 2024 at 3:08am UTC
Oasis gRPC (grpc.oasis.io) recovered.
Affected services
Updated
Nov 7, 2024 at 2:43am UTC
Emerald Web3 Gateway (emerald.oasis.io) went down.
Affected services
Updated
Nov 7, 2024 at 2:32am UTC
Emerald Web3 Gateway (emerald.oasis.io) recovered.
Affected services
Updated
Nov 7, 2024 at 2:21am UTC
Oasis gRPC (grpc.oasis.io) went down.
Affected services
Updated
Nov 7, 2024 at 1:56am UTC
Oasis gRPC (grpc.oasis.io) recovered.
Affected services
Updated
Nov 7, 2024 at 1:56am UTC
Emerald Web3 Gateway (emerald.oasis.io) went down.
Affected services
Updated
Nov 7, 2024 at 1:40am UTC
Emerald Web3 Gateway (emerald.oasis.io) recovered.
Affected services
Updated
Nov 7, 2024 at 1:40am UTC
Emerald Web3 Gateway (emerald.oasis.io) went down.
Affected services
Updated
Nov 7, 2024 at 1:35am UTC
Emerald Web3 Gateway (emerald.oasis.io) recovered.
Affected services
Updated
Nov 7, 2024 at 1:01am UTC
Oasis gRPC (grpc.oasis.io) went down.
Affected services
Updated
Nov 7, 2024 at 12:37am UTC
Oasis gRPC (grpc.oasis.io) recovered.
Affected services
Updated
Nov 7, 2024 at 12:26am UTC
Oasis gRPC (grpc.oasis.io) went down.
Affected services
Created
Nov 7, 2024 at 12:23am UTC
Emerald Web3 Gateway (emerald.oasis.io) went down.
Affected services