Gallery
Back-End Eng SRE Metrics
Share
Explore
Gallery
Back-End Eng SRE Metrics
Back-End Eng SRE Metrics

icon picker
Weekly Report

Weekly update for
7/29/2024
⁠
-
8/5/2024
⁠
⁠

Total Incidents

⁠
0
⁠
⁠

Incidents Prior Week:
0
⁠
⁠

⁠
0
⁠
open incidents!

Avg Time to Recover

⁠
⁠
⁠

Avg Time to Recover Last Week:
⁠
⁠

Avg Time to Ack

⁠
⁠
⁠

Avg Time to Ack Last Week:
⁠
⁠
This Week's Incident Timeline
9
Apr 2024
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
May 2024
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
Jun 2024
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
Jul 2024
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
Aug 2024
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
Sep 2024
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
Oct 2024
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
Nov 2024
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
Dec 2024
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
Month
TodayFit
⁠

Count of Incidents by Service
9
⁠
MTTA and MTTR by Team (mins)
9
⁠

This Week's Incidents
9
Incident Title
Time Created
Service
Latest Status
Assignee
Incident Resolved
Incident Triggered
MTTR
MTTA
URL
1
TooMany4xxErrors: The partner-file-drop service is serving an unexpected number of 4xx error codes.
7/24/2023, 10:43 AM
Partner File Drop
resolved
Chris Elion
7/24/2023, 10:43 AM
7/24/2023, 10:43 AM
20 secs
10 secs
2
KafkaDeliverySuccessCountLow: Number of successfully delivered kafka messages is lower than expected.
6/30/2023, 5:58 PM
Telemetry Ingest
resolved
Chris Elion
6/30/2023, 5:58 PM
6/29/2023, 12:04 PM
1 day 5 hrs 54 mins 22 secs
​
3
KafkaDeliverySuccessCountLow: Number of successfully delivered kafka messages is lower than expected.
6/30/2023, 5:58 PM
Telemetry Ingest
resolved
Chris Elion
6/30/2023, 5:58 PM
6/29/2023, 12:06 PM
1 day 5 hrs 52 mins 11 secs
​
4
RequestRateLow: The request rate for telemetry-ingest is lower than normal.
6/30/2023, 5:58 PM
Telemetry Ingest
resolved
Chris Elion
6/30/2023, 5:58 PM
6/29/2023, 12:06 PM
1 day 5 hrs 52 mins 13 secs
​
5
RequestRateLow: The request rate for telemetry-ingest is lower than normal.
6/30/2023, 5:58 PM
Telemetry Ingest
resolved
Chris Elion
6/30/2023, 5:58 PM
6/28/2023, 1:48 PM
2 days 4 hrs 10 mins 14 secs
​
6
KafkaDeliverySuccessCountLow: Number of successfully delivered kafka messages is lower than expected.
6/30/2023, 5:58 PM
Telemetry Ingest
resolved
Chris Elion
6/30/2023, 5:58 PM
6/28/2023, 1:48 PM
2 days 4 hrs 9 mins 57 secs
​
7
RequestRateLow: The request rate for telemetry-ingest is lower than normal.
6/30/2023, 5:58 PM
Telemetry Ingest
resolved
Chris Elion
6/30/2023, 5:58 PM
6/28/2023, 9:48 AM
2 days 8 hrs 10 mins 30 secs
​
8
KafkaDeliverySuccessCountLow: Number of successfully delivered kafka messages is lower than expected.
6/30/2023, 5:58 PM
Telemetry Ingest
resolved
Chris Elion
6/30/2023, 5:58 PM
6/28/2023, 9:48 AM
2 days 8 hrs 10 mins 15 secs
​
9
TooMany4xxErrors: The partner-file-drop service is serving an unexpected number of 4xx error codes.
6/30/2023, 5:23 PM
Partner File Drop
resolved
Chris Elion
6/30/2023, 5:23 PM
6/30/2023, 5:23 PM
17 secs
12 secs
10
KafkaDeliverySuccessCountLow: Number of successfully delivered kafka messages is lower than expected.
6/30/2023, 2:18 PM
Telemetry Ingest
resolved
Chris Elion
6/30/2023, 2:18 PM
6/30/2023, 1:35 PM
42 mins 49 secs
1 min 15 secs
11
KafkaDeliveryFailureCountHigh: Number of delivery failures for kafka messages is higher than expected.
6/30/2023, 12:21 PM
Telemetry Ingest
resolved
Chris Elion
6/30/2023, 12:21 PM
6/30/2023, 11:29 AM
51 mins 44 secs
23 mins 34 secs
12
RequestRateLow: The request rate for telemetry-ingest is lower than normal.
6/29/2023, 4:39 PM
Telemetry Ingest
resolved
Chris Elion
6/29/2023, 4:39 PM
6/29/2023, 4:06 PM
32 mins 25 secs
13 mins 36 secs
13
KafkaDeliverySuccessCountLow: Number of successfully delivered kafka messages is lower than expected.
6/29/2023, 4:39 PM
Telemetry Ingest
resolved
Chris Elion
6/29/2023, 4:39 PM
6/29/2023, 4:04 PM
34 mins 17 secs
15 mins 33 secs
14
RequestRateLow: The request rate for telemetry-ingest is lower than normal.
6/29/2023, 4:39 PM
Telemetry Ingest
resolved
Chris Elion
6/29/2023, 4:39 PM
6/29/2023, 8:06 AM
8 hrs 32 mins 48 secs
57 secs
15
KafkaDeliverySuccessCountLow: Number of successfully delivered kafka messages is lower than expected.
6/29/2023, 4:38 PM
Telemetry Ingest
resolved
Chris Elion
6/29/2023, 4:38 PM
6/29/2023, 8:04 AM
8 hrs 34 mins 42 secs
2 mins 49 secs
16
KafkaDeliverySuccessCountLow: Number of successfully delivered kafka messages is lower than expected.
6/29/2023, 4:38 PM
Telemetry Ingest
resolved
Chris Elion
6/29/2023, 4:38 PM
6/29/2023, 8:06 AM
8 hrs 32 mins 3 secs
46 secs
17
KafkaDeliverySuccessCountLow: Number of successfully delivered kafka messages is lower than expected.
6/29/2023, 4:38 PM
Telemetry Ingest
resolved
Chris Elion
6/29/2023, 4:38 PM
6/29/2023, 4:06 PM
31 mins 29 secs
13 mins 9 secs
18
KafkaDeliverySuccessCountLow: Number of successfully delivered kafka messages is lower than expected.
6/28/2023, 5:11 PM
Telemetry Ingest
resolved
Chris Elion
6/28/2023, 5:11 PM
6/28/2023, 1:46 PM
3 hrs 25 mins
46 secs
19
KafkaDeliverySuccessCountLow: Number of successfully delivered kafka messages is lower than expected.
6/28/2023, 5:11 PM
Telemetry Ingest
resolved
Chris Elion
6/28/2023, 5:11 PM
6/28/2023, 9:46 AM
7 hrs 25 mins 15 secs
36 secs
20
DBT Failure Message: Database Error in model miso_dart_lmp_prices_acfg (models/marts/feast/price/miso_dart_lmp_prices_acfg.sql) 001003 (42000): SQL compilation error: syntax error line 57 at position 4 unexpected 'select'. syntax error line 59 ...
6/28/2023, 6:50 AM
Infrastructure Operations
resolved
Alejandro Botas
6/28/2023, 6:50 AM
6/22/2023, 12:59 PM
5 days 17 hrs 51 mins 39 secs
​
21
DBT Failure Message: Database Error in model stg_ye_dscloud_yesdata__TS_GEN (models/staging/ye_dscloud/yesdata/stg_ye_dscloud_yesdata__TS_GEN.sql) 001003 (42000): SQL compilation error: syntax error line 109 at position 0 unexpected 'select'. c ...
6/28/2023, 6:50 AM
Infrastructure Operations
resolved
Alejandro Botas
6/28/2023, 6:50 AM
6/22/2023, 12:59 PM
5 days 17 hrs 51 mins 39 secs
​
22
DBT Failure Message: [0m21:46:48 Running with dbt=1.4.1 [0m21:46:49 Found 67 models, 432 tests, 0 snapshots, 0 analyses, 727 macros, 0 operations, 0 seed files, 26 sources, 0 exposures, 0 metrics [0m21:46:49 [0m21:46:53 Concurrency: 4 thread ...
6/28/2023, 6:50 AM
Infrastructure Operations
resolved
Alejandro Botas
6/28/2023, 6:50 AM
6/21/2023, 3:47 PM
6 days 15 hrs 3 mins 43 secs
​
23
KafkaDeliveryFailureCountHigh: Number of delivery failures for kafka messages is higher than expected.
6/27/2023, 10:50 PM
Telemetry Ingest
resolved
Chris Elion
6/27/2023, 10:50 PM
6/27/2023, 8:03 PM
2 hrs 47 mins 28 secs
14 mins 41 secs
24
KafkaDeliverySuccessCountLow: Number of successfully delivered kafka messages is lower than expected.
6/27/2023, 7:02 PM
Telemetry Ingest
resolved
Chris Elion
6/27/2023, 7:02 PM
6/27/2023, 3:41 PM
3 hrs 21 mins 33 secs
1 min 21 secs
25
RequestRateLow: The request rate for telemetry-ingest is lower than normal.
6/27/2023, 7:02 PM
Telemetry Ingest
resolved
Chris Elion
6/27/2023, 7:02 PM
6/27/2023, 3:43 PM
3 hrs 19 mins 33 secs
12 secs
26
KafkaDeliverySuccessCountLow: Number of successfully delivered kafka messages is lower than expected.
6/27/2023, 7:02 PM
Telemetry Ingest
resolved
Chris Elion
6/27/2023, 7:02 PM
6/27/2023, 3:43 PM
3 hrs 19 mins 13 secs
31 secs
27
DBT Failure Message: Database Error in model miso_dart_lmp_prices_acfg (models/marts/feast/price/miso_dart_lmp_prices_acfg.sql) 001003 (42000): SQL compilation error: syntax error line 57 at position 4 unexpected 'select'. syntax error line 59 ...
6/23/2023, 11:22 AM
Infrastructure Operations
resolved
Alex Jacobsen
6/23/2023, 11:22 AM
6/23/2023, 11:12 AM
9 mins 36 secs
​
28
DBT Failure Message: Database Error in model stg_ye_dscloud_yesdata__TS_GEN (models/staging/ye_dscloud/yesdata/stg_ye_dscloud_yesdata__TS_GEN.sql) 001003 (42000): SQL compilation error: syntax error line 109 at position 0 unexpected 'select'. c ...
6/23/2023, 11:22 AM
Infrastructure Operations
resolved
Alex Jacobsen
6/23/2023, 11:22 AM
6/23/2023, 11:12 AM
9 mins 32 secs
​
29
DBT Failure Message: Database Error in model miso_dart_lmp_prices_acfg (models/marts/feast/price/miso_dart_lmp_prices_acfg.sql) 001003 (42000): SQL compilation error: syntax error line 57 at position 4 unexpected 'select'. syntax error line 59 ...
6/23/2023, 11:21 AM
Infrastructure Operations
resolved
Alex Jacobsen
6/23/2023, 11:21 AM
6/23/2023, 11:05 AM
16 mins 46 secs
​
30
DBT Failure Message: Database Error in model stg_ye_dscloud_yesdata__TS_GEN (models/staging/ye_dscloud/yesdata/stg_ye_dscloud_yesdata__TS_GEN.sql) 001003 (42000): SQL compilation error: syntax error line 109 at position 0 unexpected 'select'. c ...
6/23/2023, 11:21 AM
Infrastructure Operations
resolved
Alex Jacobsen
6/23/2023, 11:21 AM
6/23/2023, 11:05 AM
16 mins 44 secs
​
31
TooMany4xxErrors: The partner-file-drop service is serving an unexpected number of 4xx error codes.
6/22/2023, 4:30 PM
Partner File Drop
resolved
Chris Elion
6/22/2023, 4:30 PM
6/22/2023, 4:29 PM
1 min 18 secs
1 min 8 secs
32
DBT Failure Message: Database Error in model miso_dart_lmp_prices_acfg (models/marts/feast/price/miso_dart_lmp_prices_acfg.sql) 001003 (42000): SQL compilation error: syntax error line 57 at position 4 unexpected 'select'. syntax error line 59 ...
6/22/2023, 12:28 PM
Infrastructure Operations
resolved
Alex Jacobsen
6/22/2023, 12:28 PM
6/22/2023, 12:28 PM
29 secs
​
33
DBT Failure Message: Database Error in model stg_ye_dscloud_yesdata__TS_GEN (models/staging/ye_dscloud/yesdata/stg_ye_dscloud_yesdata__TS_GEN.sql) 001003 (42000): SQL compilation error: syntax error line 109 at position 0 unexpected 'select'. c ...
6/22/2023, 12:28 PM
Infrastructure Operations
resolved
Alex Jacobsen
6/22/2023, 12:28 PM
6/22/2023, 12:28 PM
24 secs
​
34
DBT Failure Message: Database Error in model miso_dart_lmp_prices_acfg (models/marts/feast/price/miso_dart_lmp_prices_acfg.sql) 001003 (42000): SQL compilation error: syntax error line 57 at position 4 unexpected 'select'. syntax error line 59 ...
6/22/2023, 12:23 PM
Infrastructure Operations
resolved
Alex Jacobsen
6/22/2023, 12:23 PM
6/22/2023, 12:16 PM
6 mins 17 secs
​
35
DBT Failure Message: Database Error in model stg_ye_dscloud_yesdata__TS_FLOW (models/staging/ye_dscloud/yesdata/stg_ye_dscloud_yesdata__TS_FLOW.sql) 001003 (42000): SQL compilation error: syntax error line 109 at position 0 unexpected 'select'. ...
6/21/2023, 5:43 PM
Infrastructure Operations
resolved
Alex Jacobsen
6/21/2023, 5:43 PM
6/21/2023, 5:39 PM
3 mins 39 secs
​
36
DBT Failure Message: Database Error in model stg_ye_dscloud_yesdata__TS_GEN (models/staging/ye_dscloud/yesdata/stg_ye_dscloud_yesdata__TS_GEN.sql) 001003 (42000): SQL compilation error: syntax error line 109 at position 0 unexpected 'select'. c ...
6/21/2023, 5:43 PM
Infrastructure Operations
resolved
Alex Jacobsen
6/21/2023, 5:43 PM
6/21/2023, 5:39 PM
3 mins 35 secs
​
37
DBT Failure Message: Database Error in model stg_ye_dscloud_yesdata__TS_FLOW (models/staging/ye_dscloud/yesdata/stg_ye_dscloud_yesdata__TS_FLOW.sql) 001003 (42000): SQL compilation error: syntax error line 109 at position 0 unexpected 'select'. ...
6/21/2023, 5:39 PM
Infrastructure Operations
resolved
Alex Jacobsen
6/21/2023, 5:39 PM
6/21/2023, 5:29 PM
10 mins 5 secs
​
38
DBT Failure Message: Database Error in model stg_ye_dscloud_yesdata__TS_GEN (models/staging/ye_dscloud/yesdata/stg_ye_dscloud_yesdata__TS_GEN.sql) 001003 (42000): SQL compilation error: syntax error line 109 at position 0 unexpected 'select'. c ...
6/21/2023, 5:39 PM
Infrastructure Operations
resolved
Alex Jacobsen
6/21/2023, 5:39 PM
6/21/2023, 5:29 PM
10 mins 1 sec
​
39
DBT Failure Message: Database Error in model stg_ye_dscloud_yesdata__TS_FLOW (models/staging/ye_dscloud/yesdata/stg_ye_dscloud_yesdata__TS_FLOW.sql) 001003 (42000): SQL compilation error: syntax error line 109 at position 0 unexpected 'select'. ...
6/21/2023, 5:39 PM
Infrastructure Operations
resolved
Alex Jacobsen
6/21/2023, 5:39 PM
6/21/2023, 5:36 PM
2 mins 22 secs
​
40
DBT Failure Message: Database Error in model stg_ye_dscloud_yesdata__TS_GEN (models/staging/ye_dscloud/yesdata/stg_ye_dscloud_yesdata__TS_GEN.sql) 001003 (42000): SQL compilation error: syntax error line 109 at position 0 unexpected 'select'. c ...
6/21/2023, 5:39 PM
Infrastructure Operations
resolved
Alex Jacobsen
6/21/2023, 5:39 PM
6/21/2023, 5:37 PM
2 mins 19 secs
​
41
DBT Failure Message: Database Error in model stg_ye_dscloud_yesdata__TS_FLOW (models/staging/ye_dscloud/yesdata/stg_ye_dscloud_yesdata__TS_FLOW.sql) 001003 (42000): SQL compilation error: syntax error line 109 at position 0 unexpected 'select'. ...
6/21/2023, 5:39 PM
Infrastructure Operations
resolved
Alex Jacobsen
6/21/2023, 5:39 PM
6/21/2023, 5:38 PM
38 secs
​
42
DBT Failure Message: Database Error in model stg_ye_dscloud_yesdata__TS_GEN (models/staging/ye_dscloud/yesdata/stg_ye_dscloud_yesdata__TS_GEN.sql) 001003 (42000): SQL compilation error: syntax error line 109 at position 0 unexpected 'select'. c ...
6/21/2023, 5:39 PM
Infrastructure Operations
resolved
Alex Jacobsen
6/21/2023, 5:39 PM
6/21/2023, 5:38 PM
34 secs
​
43
TooMany4xxErrors: The partner-file-drop service is serving an unexpected number of 4xx error codes.
6/21/2023, 4:26 PM
Partner File Drop
resolved
Chris Elion
6/21/2023, 4:26 PM
6/21/2023, 4:22 PM
3 mins 20 secs
43 secs
44
DBT Failure Message: [0m23:06:39 Running with dbt=1.4.1 [0m23:06:39 Unable to do partial parsing because of a version mismatch [0m23:06:43 Found 67 models, 419 tests, 0 snapshots, 0 analyses, 727 macros, 0 operations, 0 seed files, 26 sources, ...
6/20/2023, 5:18 PM
Infrastructure Operations
resolved
Alex Jacobsen
6/20/2023, 5:18 PM
6/20/2023, 5:06 PM
11 mins 33 secs
​
45
ContainerMemoryUsage: telemetry-ingest - Container Memory usage is too high.
6/20/2023, 3:00 PM
Telemetry Ingest
resolved
Chris Elion
6/20/2023, 3:00 PM
6/20/2023, 2:56 PM
4 mins 18 secs
31 secs
46
ContainerMemoryUsage: telemetry-ingest - Container Memory usage is too high.
6/20/2023, 3:00 PM
Telemetry Ingest
resolved
Chris Elion
6/20/2023, 3:00 PM
6/20/2023, 2:56 PM
4 mins 15 secs
23 secs
47
ContainerMemoryUsage: telemetry-ingest - Container Memory usage is too high.
6/20/2023, 3:00 PM
Telemetry Ingest
resolved
Chris Elion
6/20/2023, 3:00 PM
6/20/2023, 2:56 PM
4 mins 15 secs
54 secs
No results from filter
⁠
Service
⁠

Gallery
Share
 
Want to print your doc?
This is not the way.
Try clicking the ⋯ next to your doc name or using a keyboard shortcut (
CtrlP
) instead.