Horatiu’s Blog

Visiting Japan (東京マラソンを走る)

2026-03-08T10:49:29-04:00

For years, I had wanted to visit Japan, but cycling trips in Europe always seemed more tempting and kept winning out. Then, the day before the Tokyo Marathon lottery closed, a friend convinced me to apply. I didn’t expect much, but a few months later I received the email: I was selected.

In case you can’t read Japanese, 東京マラソンを走る translates to “running the Tokyo Marathon.” What followed was a winter of training in snowy New York, a jet-lagged flight across the world, and eventually my first attempt at the marathon distance.

Ready for First Marathon!?

My running history starts in 2023, when a friend suggested we sign up for a 10K. I had never even run a 5K before, so I started a Couch-to-10K program in January 2023 with a goal of breaking 50 minutes by March. Given my cycling training, cardio was not the bottleneck, but my tendons were not primed for running. Like many beginner runners, I treated every run like a race - getting injured, taking a few days off, and repeating the cycle. Nonetheless, I hit my goal running a 49:56 in my first race!

After that, I took a pause for a few months. The week before a half-marathon, I #YOLO signed up and paid for it, blowing up and getting a disappointing 2:10. Since then, I started a more proper training program in 2024 incorporating easy runs and intervals, which steadily improved my running performance.

Fast-forward to late 2025: I was running 2–3 times per week and felt reasonably comfortable at moderate distances. Definitely not marathon shape, but something to work off of.

Training Strategy

I didn’t follow a traditional marathon training program. Instead, I devised one where I’d gradually add more mileage and cross-train on the indoor trainer bike. On February 1st, I would check-in on my progress through the Manhattan 10K where I was hoping for a sub-44. My goal was to hit ~40mpw as peak mileage.

My training calendar - I had quite good recovery from all the zone-2 volume (you can see my resting heart-rate was below 50bpm for most nights and HRV above 80ms which for me is good recovery)

The main issue in the training block was the cold and snowy New York winter. In the 2025/2026 winter, there was more than 3x the amount of snow from the previous year and almost 2x the snow of a regular winter. Icy conditions and extreme cold made it difficult to do long runs and workouts before/after work. In fact, it was so bad that the Manhattan 10K was even cancelled this year due to the cold.

Geese sleeping and enjoying the view on a Hudson River Iceberg in Jersey City. Passed by this view on every run for my training block

Lesser known running route in North Bergen above the Palisades. The roundtrip 26km+ run felt like an adventure every time, passed through numerous towns and super steep hills

From the late-nights of running in freezing weather, long runs where my water bottle would turn into a block of ice, running in 1km loops in traffic because of unplowed sidewalks - the training cycle definitely had a distinct vibe. Maybe not for everyone, but I still enjoyed it.

Overall, I was satisfied with my training and resiliency given the weather, and felt strong. I had a streak of 43 days of running/cycling, with several double-days. I still went to the gym 4x a week during the training block to lift and maintain strength. I lost ~5lbs total throughout all of this which wasn’t ideal, but my Garmin predicted time was 3:23 and Strava gave a 3:33 so I figured 3:30 could be in the cards!

Pre-Tokyo Marathon Plan

With the training and feeling prime, I flew in 3 days before the marathon … or so I thought, but it was miscalculated and I actually flew in two days before because I lost a day due to time zones (yes, I studied math in undergrad). The flight was rough, I had a window seat but couldn’t get up as much as I wanted.

Due to a snowstorm, the flight was delayed three hours on the runway. In total, I ended up spending 17 hours on the plane. After arriving, I got slightly sick - perfect timing after not getting sick for over a year! Sore throat, a lot of coughing, congestion. I didn’t have any medication, so went to Japanese pharmacies to get some things for a cold and Vitamin C. Unfortunately, nothing was in English so I struggled reading instructions with ChatGPT saying to take 3 pills 3x a day, Google translate saying 5 pills 2x a day…

It’s safe to say the trip there was rougher than I thought. The hotel room also didn’t have A/C turned on (supposedly the hotel only turns it on in April), so I could barely sleep. Doesn’t help that I was 13 hours jet-lagged either.

The day before, I had bib pickup at the expo and a half-marathon relay shakeout with my dad the night before the actual marathon. The expo was busy and crowded. The relay was quite fun, you pass around the green baton belt. You can read more about the Japanese culture of relays races, called Ekiden. Doing it sick was not great, but got it done. Waiting sweaty in the cold after my shift, didn’t help in feeling better.

Start of the Half-Marathon Relay. I did 5km of the shake-out (~5:30min/km) and my dad finished 16km (~4:59min/km). In my defense I had to run a full marathon the next day 😛

Overall, felt a bit shaky coming into the marathon. After some all-you-can-eat sushi with friends and pasta, I was hoping to feel better and sleep well the night before the marathon!

Race Day Strategy & Execution

My original race plan was to aim for 3:30, but after getting sick I had no idea how the race would go. My strategy was to take it very easy the first km, if I feel good aim for 1:45 half-marathon, and then go all out in the last half. While signing up for the race I put a 4:15 expected time by accident, putting me far back in Corral G. This posed a problem for my 3:30 goal.

After not sleeping the night before (Garmin sleep score 44), and being sick, I woke up already exhausted. Luckily, my hotel was right beside the start-line so I saved some energy from the commute. After getting to the start, I thought to use the washroom before the race, but the lines were long and I started in Corral J unintentionally. These were costly mistakes, and I lost a lot of time to weaving and getting blocked by people - especially at the water stations, where most walked and blocked access to it. On the other hand, there was mental boost for passing so many people constantly.

The first km was very crowded, but I felt surprisingly OK running. I picked up the pace after, hitting my goal of ~1:45 for the first half marathon and felt surprisingly great, and not tired at all. The crowds were incredible, and once I settled into the rhythm of the race I almost forgot I was sick!

Running around Ginza, right around the Tokyo tower (~35km)

The final stretch of the race (~40km). You can see that I had Corral G (for 4:15 expected time) but caught up with Corral D (3:30) towards the end of the race

Picture at the finish with the Tokyo Marathon 2026 medal

I maintained a fairly low HR (~159bpm average for the half marathon). After 30km, I started to feel knee pain, which I never felt in training. My feet swelled up, and started pressing against the tip of my Vaporflies, which resulted in black nails and was painful. Come 37km, I started to get dehydrated since it got warm (66ºF) and I was skipping water stations since it would cost too much time.

However, I powered through and felt very strong at the end. The last km was on a cobblestone road that was narrow and felt like a speed-tunnel, as the finish was at the very end and since I had a lot of energy left. Starting so far back meant I spent most of the race weaving through runners - it cost time, but there was also something strangely motivating about constantly passing people for 40 kilometers. Crossing the finish line, after months of winter training, felt surreal - the kind of moment that makes all the miles feel worth it.

Felt quite good until 30km. It is quite common to say that the marathon really only starts at 32km, where you hit the real bottleneck. For me, it was knee pain, and my feet swelled up and pressed against my Vaporflies which resulted in black nails. My aerobic base training and fueling were seemingly on-point otherwise

Overall, I would consider this a successful first full-marathon, having avoided the wall and no major issues! I got an official time of 3:40:06. While I missed my target time by ~10 minutes, given the circumstances I was still very satisfied.

Lessons Learned & What Worked

Lessons learned:

Do not underestimate jetlag - long flights, time-zone, changes in culture/cuisine all add fatigue and stresses the body so recovery is not optimal. While aiming for PRs, local races might be better picks.
Skew towards slightly larger shoes for races - feet swell up during longer races. While size-10 fits me normally, in this race it was too small.
Pick faster corrals - it is always easier to select a faster time than you think and start further back than the converse.

Some things that worked well for me:

Cross-training helps - I did relatively low mileage, but significant base training from cycling helped in maintaining fitness for running. Cycling is low-impact so less injury risk.
Gel fueling strategy - I had 6 Gels which were solid (taken at 5km/12km/20km/25km/30km/35km), did not bonk.
Taking it easy - I had a low HR throughout the race (averaged 159bpm) which meant I didn’t blow up in the later half - conservative pacing was worth it.

Post-Race Touristy Adventures

After the race, I had some time to recover from the cold and do sightseeing with my parents in Tokyo and surrounding regions.

Taking the Shinkansen from Tokyo, you can reach Kyoto in ~2 hours. It was a bit pricey but worth it. We just did a day-trip, but staying a night wouldn’t be a bad idea to see other nearby things like Nara, which we missed unfortunately.

Kiyomizu-Dera in Kyoto

Fushimi Inari-taisha Shrine in Kyoto. Tip, it’s much less crowded as you go further up!

Making the most of rainy days by taking this timelapse at the Shibuya crossing (from the 2nd floor Starbucks)

Bamboo forest at the Hokokuji Temple in Kamakura, ~1 hour away from Tokyo

Night view from the Government Building in Shinjuku. They had a piano competition that night, with some really talented Japanese musicians playing (YouTube link)

Retrospective

I would consider this trip a success. Looking back, I probably left a few minutes on the table due to sickness, the corral mistakes, shoe mis-sizing, and jet lag. But for a first marathon, under less-than-ideal circumstances, I’m happy with how it went and all the training build-up for it.

As a plus, I got to do a lot of sight-seeing and try Japanese food, culture and sights with friends/family. I found the trip both energizing, relaxing and exciting all at once. I had high expectations coming in, and Japan absolutely did not disappoint. Now I’m already thinking about the next marathon, maybe NYC? ;)

A week cycling in Girona and Catalonia

2025-09-13T12:49:29-04:00

In September 2025, I had the pleasure of visiting Girona, Spain. Girona is a popular cycling hub where numerous famous athletes including Jan Frodeno reside because of the excellent weather, roads, and variety of routes available.

Training Leading up to Girona

I did not do any specific training for this trip. The year started off rough after catching COVID19 in January and having to take off several weeks as I followed the protocol for returning to exercise. Although I was almost entirely asymptomatic, it took several weeks for all performance metrics to fully recover.

After that, I had a steady, focused training block of several months where I reached my highest level of fitness to-date. This year, I was able to check-off several longstanding goals such as a VO2 Max of 60, under 1 hour of Alpe du Zwift (58:21), under 20 minutes on Bear Mountain (19:52), and hit my FTP goal of 3.7W/kg including holding 200W in Zone-2.

In conjunction with strength gains at the gym, I would consider my buildup leading to this trip as prime.

Trip Strategy

The plan was simply to complete all the notable routes near Girona. In total, I planned for six rides, and since I was staying 7 days I could afford 1 rest day. This plan was distinct from the Bormio or Mallorca trips as there was not a singular flagship ride, but rather a series of moderate rides. This meant recovery and energy preservation was key as I had many back-to-back 100km+ rides with considerable elevation gain.

Figure 1: GPS overlay of the different cycling routes

Day #1: Els Àngels

71.11km, 1003m Elevation Gain

The first ride was doing the local Els Àngels loop, which meant doing both Els Àngels (Category 2) and Alt de Santa Pellaia (Category 3) on the way back. The views were not that impressive, but the descent down the eastern side of Els Àngels was one of my favorite descents I’ve ever ridden of the winding nature, excellent road quality and good visibility around corners.

Day #2: Mare de Déu del Mont

125.97km, 1905m Elevation Gain

This ride was the longest ride with the most climbing. The centerpiece of this route is the Mare de Déu del Mont climb which is a category HC climb at 5.9% grade and 925m of elevation gain. While the incline doesn’t seem challenging, there are a few flat sections, which lead to a deceptively low average gradient.

The final 1.5km are at a solid 9.7% which includes several shorter sections above 14%. Towards the top the trees begin to fade and make way for rock formations and stunning views of the Pyrenees. If you compare it to Mallorca, this climb is harder than those found on the island including Sa Colobra.

Figure 2: One of many hairpin turns on the climb

Figure 3: The summit at Mare de Déu del Mont

Day #3: Rest Day Run

10.34km, 105m Elevation Gain

This was a rest day, where I went for a run instead. A few days after I return to New York, I had to run the NYRR Bronx 10M race so I didn’t want to lose all adaptations.

Figure 4: Running trail on the outskirts of Girona

Day #4: Tossa de Mar (Coastal Loop)

120.07km, 1668m Elevation Gain

This route was very scenic, and included riding along the Mediterranean coast. There were a lot of short ups and downs, but no major climbs except for Els Àngels des de Madremanya which was harder than climbing it from the eastbound direction.

Figure 5: View of Tossa de Mar and the Mediterranean coast

Figure 6: Picture of my dad riding behind me on the windy roads

Figure 7: Overlooking another cliffside along the Mediterranean coast

Day #5: Sant Hilari Sacalm and Susqueda Dam

109.36km, 1590m Elevation Gain

This route going westward from Girona featured both the Sant Hilari climb (Category 1) and the Susqueda Dam. The ride was unique in that the majority of the route felt remote and in a mountainous region.

The unfortunate part of this ride is that I burned my right calf after descending to the dam; I accidentally touched the brake disk when I got off my bike and it burned my skin right off. It was quite painful in the moment but no option apart from moving on!

Figure 8: View of the Susqueda Dam after descending from Hilari

Figure 9: Eastern viewpoint of the Susqueda Dam

Day #6: Rocacorba

73.24km, 1206m Elevation Gain

The final route was centered around Rocacorba, generally regarded as the hardest climb in the region due to its varying gradients, and is a Category 1 9.82km climb with an average gradient of 8.4%. The views were not particularly impressive, and on the descent I had a minor crash where I overcooked a corner and could not stop as the road surface was wet.

I did a tempo effort up this climb at 220 W. It wasn’t particularly quick, but considering the accumulated fatigue over the past few days I was satisfied with the results. I got a segment time of just over 56 minutes.

Figure 10: Elevation Profile of the Rocacorba climb

Figure 11: Summit of Rocacorba overlooking towards Girona

Apart from Rocacorba, the route started with some gravel and passed Banyoles Lake which was beautiful.

Girona & Barcelona

Aside from cycling, I also had the opportunity to explore Girona and spend a day in Barcelona. Girona is a cute medieval town which was featured in many movies and TV shows including Game of Thrones.

There are a lot of cycling/triathlon oriented shops and cafes, there’s even billboard advertisements for conferences by Shimano. Barcelona pleasantly surprised me, the beaches and architecture combined with the clean and wide boulevards were a pleasure to sight-see.

Muralles de Girona (city walls) overlooking the Cathedral of Saint Mary

Retrospective

This trip went exactly as planned and Girona delivered. While I don’t plan to return, the trip met my expectations and was an enjoyable experience. If I were to change something about my trip, I would have done more research on food options as I did not enjoy Catalan food as much, and I found the dining hours unusual — dinner often started after 8pm.

Looking forward to future trips, there’s a lot planned for next year so stay tuned ;)

YouTube Descents

Feel free to watch some descents on my YouTube channel!

Visiting Great Smoky Mountains National Park

2024-10-21T19:49:29-04:00

In October 2024, I had the pleasure of visiting Smoky Mountains National Park with several friends. The park, situated in Tennasse and North Carolina, is the most visited national park in the United States attracting more than 13.3 million visitors annually.

Mount LeConte

Our first significant hike was Mount LeConte, which is the highest peak completely in Tennessee standing at 6593ft (2010m), and has the highest relief east of the Rocky Mountains at 5301ft (1616m). We did the Alumn Cave trail, and reached the main summits including High Top and Myrtle Point.

The initial fall views after sunrise were fantastic, but after there weren’t many views until closer to the summit. Compared to a trail like the Grand Canyon or those at Zion, there was not much to keep you entertained on the hike but the views at the top are still worthwhile.

Figure 2.1: View of the Smoky Mountains

Figure 2.2: View from the Alum Cave Bluff

Figure 2.3: View from the LeConte Lodge which is the highest inn that provides lodging in Eastern US

Mount Cammerer

The second significant hike was Mount Cammerer via the Lower Gap trail. The trail intersects with the Appalachian trail, and reaches a peak of 4928ft (1502m). To my surprise, there was snow on the trail in mid-October!

Figure 3.1: Snow along the trail up to the summit of Mount Cammerer

Figure 3.2: More snow along the tail; it was quite cold near the top

Figure 3.3: Interesting firetower at the summit with spectacular views

Figure 3.4: Panorama view at the summit

Cades Cove

Cades Cove was a simple drive-through loop with nice scenic views. After looping around for almost an hour, we eventually got stuck in traffic, and the most exciting part of seeing a black bear appeared!

Unfortunately, the Bear did not turn out great on camera :)

Figure 4.1: Black bear who was not in a very photogenic angle

Cligman’s Dome

Cligman’s Dome is the highest point in the park and is also one of the easiest trails. There is a parking lot nearly all the way to the summit, with only a short walk up a ramp. Since it was below freezing, the ramp up to the dome was pure ice and extremely dangerous to go down, which makes this very short hike the most prone to falling by-far.

Figure 5.1: The ramp to the highest point in the park. At the end it was only half icy, but for the rest it was pure ice and extremely slippery

Figure 5.2: View of the park after sunrise

Figure 5.3: The sunrise was enjoyable with a nice panoramic view

Forbidden Carvens

Forbidden Caverns is located near the Smoky Mountains and were originally used to make moonshine. The cave descends several hundred feet underground, and remains a constant temperature of around 58ºF all-year round. It was an interesting informative tour.

Ziplining

I never went ziplining before, so the prospects of going 500ft above the ground clipped to a steel cable seemed rather terrifying. After doing the trial run, it didn’t seem too bad, but after going to the longer runs it quite scary!

For the longest run, I was told to run a bit to get momentum and not to brake at all otherwise I would get stuck halfway. Of course, I did not follow the instructions, and nearly got stuck in the middle.

As a punishment, on the way back the instructor shaked the zipline steel wire while I was ziplining through it, and I thought that the end of me (my heart rate hit 135bpm while my resting heart rate is usually less than 50bpm) - never doing this ever again!

Figure 7.1: Ziplining backwards with views of the Smoky Mountains

Retrospective

This was a fairly light vacation in terms of physical activity. While the lodge we stayed at was fantastic, my impression was the nearby towns (such as Gatlinburg) didn’t quite suit my tastes. They felt suburban and very car-centric, with a lot of tourist traps and poor facilities for activities like running. In fact, to do extra mileage for running I had to do hill repeats in my lodge’s parking lot as nearby roads didn’t have sidewalks.

The park itself was really nice with the fall foliage, however the views did not compare to west coast national parks such as Zion or Grand Canyon. While I don’t regret visiting the park, for future trips I will favor west coast national parks instead (i.e., Yellowstone or Glacier).

A week cycling in Mallorca and Serra de Tramuntana

2024-06-15T19:49:29-04:00

In June 2024, I had the pleasure of visiting Mallorca, Spain. Mallorca is the largest of Spain’s Balearic Islands. Known for its beach resorts and limestone mountains, it is a popular cycling destination and frequently marked as “cyclist’s paradise” because of the warm weather, fantastic food, varied terrain, and well-maintained roads.

Getting sick right before the trip

While I was in relatively good shape (albeit most of my training was for running, as the Brooklyn Half Marathon had recently happened), I got sick right before leaving New York. My initial symptoms appeared like a cold, so it didn’t present as a blocker for going on the trip.

I rarely get sick, but unlucky timing? More on this later …

Figure 1.1: Garmin HRV insights leading up to the cold. HRV (heart rate variability) measures the time variation between consecutive heart-beats. It is an indirect measurement of physiological stress, and when the body is under stress the sympathetic branch is more active, leading to increased heart rate and beats in a more regular rhythm hence decreasing HRV. You can learn more here

Trip Plan

Situated right outside Port de Pollenca, the plan was to do several rides with my dad, and spend the off-days with my mom on the island. The main routes of interest were Cap de Formentor, a flagship ride from Andratx to Port de Pollenca, and Sa Colobra. Since I was sick, exceeding the aerobic heart-rate (Zone 2) was undesirable so my intention was to cruise most climbs if possible until my HRV fully recovered.

Figure 2.1: Hotel in Port de Pollenca, the food was great and within close distance to most famous routes. The bike was a Trek Emonda which worked quite well and had comfortable 11-36 gearing

Day #1: Cap de Formentor

52.06km, 1006m Elevation Gain

The first ride was going to the famous Formentor Lighthouse. The ride was moderate in difficulty, with a few category 3 and 4 climbs. It was very scenic, and had a good variety of geographies while being secluded from traffic. I also almost crashed into a goat on the way back!

Figure 2.2: Hair-pin turns from behind Cap de Formentor, at the end of the island

Figure 2.3: Myself looking at Cala Figuera

Figure 2.4: View overlooking Alucdia Bay from the watchtower

Day #2: Ermita la Victoria

28.42km, 408m Elevation Gain

This ride was a fairly easy spin to a millitary base on the other side of the Alcudia Bay. There were a few sharp sections on the climbs, but otherwise a light ride.

Figure 2.4: View overlooking Alucdia Bay from near the millitary base east of Alcudia

Day #3: Andratx to Port de Pollenca

140.12km, 3029m Elevation Gain

The flagship ride was a one-way trip across the northern mountaineous part of the island. It started at 5:30am where we took a bike shuttle from Port de Pollenca to Andratx. From there, riding across the coast and stopping at several ports added significant elevation and difficulty. Some interesting spots included Soler and visiting the historic town of Valldemossa.

There were several notable climbs/mountains to conquer:

Puig Major (720m Elevation Gain, 6.0%, Category 1)
Port des Canonge + Col de Claret (477m Elevation Gain, 5.4%, Category 2)
Port de Valdemosa (366m Elevation Gain, 7.5%, Category 2)
Col de sa Gramola (268m Elevation Gain, 5.0%, Category 3)
Col de sa Bastida (168m Elevation Gain, 6.2%, Category 4)

I almost skipped the ride because I couldn’t sleep the night before and felt sicker since I started to cough consistently. At that point, I wasn’t sure if I caught COVID-19 or some other virus. Nonetheless, I continued and tried to keep my pace within Zone-2 the entire ride.

Figure 2.6: Elevation chart for the ride. There were many moderate climbs, but apart from Puig Major nothing was similar to the climbs you would find the Alps

Figure 2.7: Sunrise at Port de Pollenca while waiting for the bike shuttle

Figure 2.8: Viewpoint from Puig Major, the highest elevation mountain pass in Mallorca

Overall I would consider this ride a succeess, especially considering I was sick while doing it. Surprisingly, I wasn’t tired at the end like in my previous 3000m+ elevation gain ride in the Stelvio Pass from the year prior.

Day #4: Local Villages

58.26km, 298m Elevation Gain

A relaxed recovery ride included a local route through several villages (Santa Margalida, Muro, Sa Pobla).

There wasn’t anything super notable, I found the ride quality slightly mixed with quite a bit of traffic on some roads. Generally, I would much prefer the mountaneous route from earlier routes but nontheless is was a good recovery effort.

Figure 2.9: Typical street connecting villages, in this case Muro and Sa Pobla

Day #5: Rest Day, Sicker

This day I took a rest day and went on a boat tour with both my parents. Unfortunately, I got sicker and started having a fever, and my cough got a lot worse …

Day #6: Skipped Final Ride

For my last day, I had to unfortunately skip doing the Sa Colobra climb because of fever and excessive coughing. I tested negative for COVID, and three days later after arriving in New York I learned I had developed pneumonia and had to take antibiotics. My dad still completed the ride though, which had a lot of overlap with the Day 3 ride except for the Sa Colobra climb itself.

Retrospective

The trip started on a rough footing after catching a cold, but overall still completed most climbs until having to skip my last ride due to illness. It was a relaxing trip, and the overall community and vibe on the island was very friendly towards cyclists.

What went well

Completed most intended rides, including my hardest flagship ride covering the Serra de Tramuntana
Food was excellent, weather was perfect, rental road bike was pretty good
Great variety of activities to do in Mallorca, including visiting the beach and other areas on the island
Enjoyed time with my parents

What could have gone better

Getting really sick affected the quality of my rides and overall satisfaction of the trip; the flight back was brutal with excessive coughing
There weren’t any challenging climbs such as the Stelvio Pass, which made the vistas not as impressive in comparison
Descents did not appear as impressive on my GoPro because of the aforementioned point

Overall, I am glad I went, though I wouldn’t go again because there are other more challenging climbs to do elsewhere in Europe. The only event I would potentially consider going back for is if I am already in Europe and wanted to complete the Mallorca 312.

YouTube Descents

Feel free to watch some descents on my YouTube channel! Apologies for the coughing in some of the clips.

A week cycling in Bormio and the Alps

2023-07-31T19:49:29-04:00

In July 2023 I had the opportunity to spend a week in Bormio, Italy and go road biking in the Alps. The northern region of Lombardy is filled with iconic climbs and history.

For reference, road cycling climbs are often characterized by several properties:

Altitude - the difficulty in riding increases as altitude increases due to the lack of oxygen. For example, beyond 2500m there is ~25% less oxygen available compared to sea level.
Gradient - the average slope of the climb. Inclines ranging from 3-5% are considered mild, 5-7% moderate and 7%+ is generally considered difficult.
Distance - the majority of climbs are between 10-40km from beginning to summit.

The Tour de France provides a classification for climbs, going from Category 4 (easiest) to Category 1 (hardest), with a special category HC (Hors Catégorie) reserved for climbs originally “unpassable by car”.

Day 1: Lago di Cancano

28.32km, 762m elevation gain (Category 2)

To warm-up and test the rental bike, a local ride up to Lago di Cancano was worthwhile. The ride was relatively short but it featured a Category 2 climb reaching 1938m elevation with a 6.9% grade. Notably, even the easiest ride around Bormio is steeper and nearly twice the elevation gain of Bear Mountain in NY.

Day 2: Stelvio Pass + Umbrail Pass

105.14km, 3163m elevation gain (2x HC climbs)

The Stelvio Pass is the highest elevation mountain pass in the Eastern Alps (2757m tall), and the 2nd overall standing 7m below France’s Col de l’Iseran. It was built in the 1800s by the Austrian empire to connect Lombardy with the rest of Austria. It is featured regularly as the Cima Coppi of the Giro D’Italia.

There are three ways to get to the summit - riding the Stelvio from Bormio (Italy), Prato (Italy), or Umbrail Pass from Val Müstair (Switzerland).

Figure 2.1: In this ride, I climbed from Bormio, descended Umbrail Pass into Switzerland, then went back up from Prato to the summit and then descended back to Bormio.

Aside from a short section with 13-14% slope it was fairly managable. I managed to do it without stopping except for a small photo break.

After doing 40 hairpin turns, taking a left-turn on the SS38 will lead to the Umbrail Pass which is the highest elevation mountain pass in Switzerland descending down to the region of Val Müstair. After arriving at Val Müstair it was flat for a short period, I ate a quick sandwhich and then prepared for the final monster which is the 1893m climb to the Stelvio summit from Prato.

In combination with fatigue from the earlier climb, I found this HC climb absolutely brutal.

Things took a turn for the worse at around 14km into the climb once reaching the forest area where gradients hovered around 8-10% bend after bend. After passing 2000m elevation the altitude really sunk in making it even more miserable, and my right knee was hurting on each pedal stroke. Unfortunately, I had no option but to continue because my hotel was on the other side of the mountain…

Getting out of the forest and seeing the hairpin turns spiraling up to the Stelvio peak made summitting seem like an unachievable feat in the moment. Even pushing 150 watts (a relatively light power output for me) was difficult due to fatigue and lack of oxygen. Though at the end, after completing the 48th bend and 3.5 hours of suffering to reach the summit, words cannot describe the experience of joy (and relief).

Day 3: Sondalo

42.46km, 550m elevation gain

This was a recovery day, featuring an easy ride to Sondalo and back through the country roads. It was relatively flat, with one Category 4 climb (7.8% with 101m elevation gain) but otherwise easy.

Day 4: Bormio 2000

17.3km, 577m elevation gain (HC -> Category 2)

Originally Day 4 was supposed to be Gavia pass, but unfortunately there was a thunderstorm scheduled at noon. The Bormio 2000 climb was nearby, which was also an HC climb with an 8.7% average gradient and 817m elevation gain, so that was promising. Sadly, the weather took a turn for the worse even earlier so I had to turn around a couple of minutes before reaching the summit.

Luckily I got back to the hotel minutes before the storm started, so that was good.

Day 5: Passo di Gavia

51.82km, 1422m elevation gain (HC climb)

The weather improved in Day 5 so took an attempt at Gavia Pass - the 10th highest elevation pass in the Alps reaching 2621m elevation. It connects the Sondrio in the north with Ponte di Legno in the south.

The climb wasn’t overly difficult, starting off rather mild until Santa Caterina. Afterwards, it became more challenging because of short pitches that were often 10-14%. As a prize there was a nice view at the end :)

Recap & Forward Look

These were an enjoyable set of rides, and the views were worth the pain. If I were to do it again, I would take into consideration:

Bringing my own road bike - firstly the rental was expensive and also worse quality than my personal road bike (Cervélo S5). Second, it put a lot of pressure to ride everyday to make the most of the rental when I could have hiked on off days to recover better from rides.
Attempt Mortirolo - while not as high elevation as the other climbs, Mortirolo is another one of the famous climbs that is often considered extremely difficult due to its ~10.5% steep gradient. Lance Armstrong said in an interview that it was the hardest climb he ever did.
Study the descents beforehand - during the descent from Bormio 2000 I went into a hidden hairpin turn at ~52km/h and skidded off the road (luckily didn’t fall off a cliff…) - it is difficult to memorize the map but knowing key corners can help to avoid excessive or lack of braking before corners.

Next year the plan is to attempt Mount Evans and Pike’s Peak in Colorado. They are the highest (and 2nd highest) elevation paved mountain peaks in North America surpassing 4000m altitude, so it is an exciting challenge to train and look forward to!

YouTube Descents

I recorded descents for all the climbs and overlayed the Strava data - enjoy.

Day 2 Descents (Stelvio + Umbrail Pass)

Day 5 Descent (Gavia Pass)

Visiting Grand Canyon, Death Valley, and Zion National Parks

2023-04-28T19:49:29-04:00

In April 2023 I flew out with friends to spend a week visiting several National Parks. We started in Pheonix and visited the Grand Canyon, Las Vegas, Death Valley, and Zion.

Grand Canyon

First stop was the Grand Canyon, where we planned to hike down to the Colorado River and back in one day, otherwise known as the “Rim to River to Rim” hike. The hike itself is ~14.83mi and 4675ft elevation gain, starting from South Kaibab and going to Bright Angel along the Southern Rim of the Grand Canyon.

Figure 1.1: Warning sign at the trailhead of South Kaibab, stating to not hike down to the Colorado River and back in one day, but we chose to ignore it. More on this later…

We arrived at the trailhead by around 5:00am local time, before sunrise, where we patiently waited to start the hike. The trail was nearly empty at the time!

Figure 1.2: View from near the Ooh Aah Point (6660 ft elevation)

Descending down the South Kaibab trail was enjoyable because it is a fairly gradual slope, so it is not as hard on the knees. Different sections of the descent brought unique views and perspective.

Figure 1.3: View around 2 hours into the descent on the South Kaibab trail, near Cedar Ridge

Figure 1.4: The Colorado River at the bottom of the Grand Canyon. The water wasn’t the cleanest!

Unlike a mountain summit where if the difficulty climbing presents to be too difficult you could elect to turn around early and descend, here that isn’t an option. Furthermore, it is hot at the bottom of the canyon.

I didn’t have any issues going back up, but my friends found it to be a streneous effort. I went ahead and left two of my friends to go up together behind me. Unfortunately, one of them bonked (ran out of glycogen), resulting in dizziness, nauseau, and fatigue.

This quickly became a pressing issue because as I was waiting at the top, it was getting dark and my friends were nowhere to be found! One of them eventually made their way up, but my friend who bonked was still in the Canyon. Luckily, he eventually got cellular signal closer to the top of the Canyon and we were able to get into contact with him. He made it right as it was getting dark.

Figure 1.5: Near the summit after climbing up Bright Angel. To my surprise, there were still snow and ice patches near the top (which would make sense considering it’s at nearly 6750ft elevation)

Overall, the Grand Canyon was a great experience and doing the “Rim to River to Rim” hike showcases a wide array of scenic views and sceneries that make it worthwhile effort!

Las Vegas

As an intermediate to the next leg of the trip, we drove to Las Vegas as our new “headquarters”. Personally, I did not find Las Vegas very enjoyable, but some of the shows were moderately interesting. The main turnoff for me was that smoking was common everywhere, and to get into our hotel we passed through a casino that was smoke-filled…

Figure 2.1: The Welcome to Fabulous Las Vegas sign. It was a bit out of the way, though naturally it was on the edge of the city.

Figure 2.2: Quick stop by Hoover Dam which was along the way to Las Vegas

Death Valley

We then ventured over to California to checkout Death Valley. This day trip showed some interesting view-points. It wasn’t that warm in California as advertised.

Figure 3.1: Zabriskie Point

Figure 3.2: Badwater Basin - the lowest point in North America (282ft below sea level) comprised mainly of sodium chloride (table salt)

Figure 3.3: Sand Dunes

Figure 3.4: Dante’s Point overlooking the Badwater Basin (5575ft)

Zion

Zion has a series of flagship hikes including “Angel’s Landing” and “The Narrows”. Unfortunately, we didn’t have a reservation for Angel’s Landing and the water was not in good condition for “The Narrows”. This wasn’t an issue, as there was still plenty to visit in our two days at the park!

First, we did the Observation Point via East Mesa Trail hike. This is arguably the best viewpoint at the park, though due a rockslide you cannot hike from the base of Zion but go through the East Mesa trail instead.

Figure 4.1: View from observation point. You can see Angel’s Landing

The actual hike was not very enjoyable. There was a lot of snow melt, which made it impassible by car to the trail-head so we endured an extra 4.3 mile on the hike because we had to park at the Zion Ponderosa ranch, while making the actual traversal tricky. It was relatively flat with no view-points until the very end.

Figure 4.2: One of many cars that failed to make it to the trail-head. There was a truck waiting on the private road that charges $1000 to tow any stuck cars (the day we went was quite profitable)

The rest was a constellation of smaller hikes we chained together, including Pa’arus, Emerald Pools (low/mid/upper), and Riverside Walk.

Figure 4.3: Sunrise overlooking the Virgin River after running ~5K from Springdale (and the Pa’arus trail). There were many wild animals before sunrise, running in the dark was kind of sketchy

Figure 4.4: Lower Emerald Pools

Figure 4.5: Upper Emerald Pools. The hike was light but the views paled in comparison to Observation Point, and it was very crowded. Some parts of the hike had a queue…

Conclusion

The trip was an exciting turn of hikes across many picturesque canyons and vistas. However, this only scratched the surface, with plenty of trails and parks left to visit in the area such as Bryce Canyon and Antelope Canyon … so a lot to look forward to!

Living in Toronto for Eight Months (COVID-19 Edition)

2020-12-31T18:49:29-05:00

What happened?

If you asked me a year ago I would be pretty shocked to see this title on my blog, knowing that I was not supposed to be in Toronto for even a single month in 2020. As you are all probably aware, COVID-19 led to many internships and school terms turning virtual and remote, and I followed suit with this.

Luckily, aside from location and being remote I wasn’t very impacted, and I am fortunate to have a safe place to live and continue my studies along with my parents at home.

April - May 2020

This period marked the gap between my Facebook and Google internships. During this time, I received my first full-time return offer from Facebook which was very exciting, and I could be rest assured that despite the pandemic taking place, I could still secure full-time employment at a top choice company for me.

I also had some spare time to explore numerous side-projects. Several notable projects include:

1) Exploring the field of Real-Time Communication at large, which originated from a report I wrote for school in relation to my term at Facebook, and later I greatly expanded it out of personal interest. Feel free to download the paper.

2) Further learning of React and Selenium web-scraping to create a data visualization site for WaterlooWorks statistics. The project involved scraping data off of Waterloo’s job board, cleaning the data and placing it on a CDN along with making the responsive site. This is my first mobile responsive site and I’m fairly proud of it. Check it out here!

3) Creating the initial services for a major overhaul of my personal site and side-projects.

I will not go into details here yet (stay tuned in 2021!) but the essence is that I am working on creating my own Raspberry Pi farm to host all my side-projects, while having failover onto AWS. This distributed cross-cloud and micro-services architecture was motivated to allow me to reduce costs on AWS EC2 instances, while maintaining reliability and flexibility. It is an ambitious project that I plan to continue for fun on weekends once I start full-time.

During this time period, I focused on creating the low-level C++ server that leverages Boost.asio to create a multi-threaded HTTP reverse-proxy and handler for different types of requests, allowing for rerouting and redirection capabilities similar to Apache or NGINX. The multi-threaded server is capable of handling around 1350 requests per second on my 2012 MacBook Pro, and is configurable to most of my use-cases. The bulk implementation was completed, alongside a basic telemetry system with heart-beats that provides analytics on particular service health.

May - August 2020

Google Internship

This period defines my Google internship on Firebase SDKs. While my last co-op was unfortunately virtual, the Google and Firebase culture certainly bled through and it was an absolute joy to work at such an extremely Engineering focused company. Special shoutout to my host and everyone else at Firebase!

During my term at Firebase I focused on the server-sided Admin SDKs. In my first month I built and shipped the ability to bulk import users into Firebase using the C# SDK. The remainder of my term focused on designing and prototyping the next-generation Firebase Node.js Admin SDK.

In particular, I researched best-practices on TypeScript (such as ECMAScript module export edge-cases) along with lessons learned from other SDKs to paint a picture on how the future SDK could look like. After careful evaluation of several designs, with heavy support of my host and members of adjacent teams, alongside 40+ pages of design documentation and several prototypes, I am proud to confirm that the formal proposal was approved (alongside some minor changes) and sets the future overarching design for the most popular Firebase server-sided SDK.

Alongside writing lots and lots of documents and prototypes, I started a new initiative to auto-generate typings from source. Previously, the Node.js SDK would have a separate d.ts typings file for all TypeScript types manually annotated, which was required due to complex monkey-patching cache logic used to lazily load modules. This process is error-prone (I ended up finding and fixing >20 bugs related to this), and keeping them in sync can be non-trivial.

However, with some careful implementation, I refactored the majority of the SDK to auto-generate typings along with adopting several new open-source tools such as API extractor and reviewing the documentation generators to play nicely with the new typing scheme. Shortly after I left, the remaining work was completed and as of today all typings are auto-generated in the Node.js SDK.

Road Biking + Tennis

During the 2020 cycling season, I managed to bike 3,786.3km across 60 rides, making it furthest distance I biked thus far in a year. Nearly every weekend, I’d ride 115-140km with my dad to Oakville and back, which was a significant contributor to this. Special shoutout to my friends as well who I went on group rides with!

My previous cycling distance record was 209km set the day after graduating high-school. This year, I broke that record by riding 245.6km with an average speed of 28.0km/h from Toronto to St. Catherines and back.

Of course, I could not use that as an excuse to not do my annual Niagara Falls bike ride, so here it is! The following ride was pretty cute as well, thought it’d mention a shoutout. If you want to do that route, keep in mind the east side is 10x more photogenic than the west.

In terms of other sports, I played tennis twice per week with several friends, and just went on regular walks.

YouTube (Learn in 5 Minutes Series)

If you subscribe to me on YouTube, you would likely be aquainted with my “Learn in 5 Minutes” YouTube series. The objective is/was to provide short clips introducing viewers to a topic sufficiently for obtaining background in it. Every week I would post a video either related to algorithms or software engineering.

During the summer (and a bit into fall) I created 27 videos spanning numerous topics including number theory algorithms (sieves, modular combinatorics) and in-depth view at software engineering concepts such as HTTP header compression (HPACK/QPACK). Feel free to visit the repository repository or website.

Given my experience with YouTube, I expected two things:

I will receive very few views at the beginning, particularily because my videos were traditionally scattered and my subscribers are saturated across various interests
The occasional video will rank well, and consequently consistently receive views

And turns out my hypothesis served correct so far. Broadly speaking, I have two types of videos: either it is consistently growing in views or it is dormant. Unfortunately, the vast majority of videos are in the second camp, but some still receive views and I am optimistic that they will receive more views later on.

This entire project was extremely time consuming, and occasionally I wondered “is it worth it?”. For many of the topics chosen, I did not have expertise before the videos, so I would need to familiarize myself with the concepts to deliver a presentation, create the presentation, record, edit and upload. To upload at a cadence of one video per week, I would need to spend at least 6hrs/week creating videos.

I will not hide the fact that delivering content every week became a stressful ordeal at times.

However, the answer to this question is still yes. Even in the case of nobody watching the videos, at the very least I learned something new. All the topics in my videos are topics that I was always interested in but did not have the time to learn them comprehensively, and through the videos it motivated me to take a deeper look. Reading through the QPACK draft because there is very little online documentation on the bleeding edge of the standard takes some level of dedication. Secondly, the views are bonus, and I am glad when my viewers learn something new from my videos :)

I’m a fairly firm believer that building fundamentals is important, and this was an example of this. I am glad to confirm for more than half the videos, I already applied the particular knowledge gained in some area (Google conversion interview, univeristy course or casual conversation). My LZ77 compression video helped a friend with her interview (coincidentally that popped up!), HTTP XSS attacks were a core focus in CS 458 (security at Waterloo) and more. Moral of the story: learning is never a waste, and sometimes wish more people took this mindset instead of simply cramming and forgetting (i.e., Leetcode or school courses).

September - December 2020

This period marks my 4A CS term. There is not much to outline for the academic term. As with many academic terms at Waterloo, my schedule quickly got busy as I had to conquer assignments from multiple classes with occasionally tricky deadlines. The school term for me was not too difficult, although it was time consuming because I took two “bird” courses that ended up having lots of essays to write.

I will outline one strategy that I used to keep myself at ease during the term, which was creating a structure during my days and trying to stick with it. Every night (morning?) at 12:00am I would make a TODO list for the next day, broken down into hourly chunks with goals. At the beginning of the term, I outlined hotspots with heavy assignment loads and ensured that I would start working on those assignments ahead of time.

Forward Looking at 2021

Without a doubt, 2020 was a strange year. Next year will be significant for me as it marks the ending of my undergraduate time at the University of Waterloo, and the beginning of my full-time career as a Software Engineer. I am hoping to end off strong at Waterloo, have some rest, and start strong at Facebook in a team that will align with my interests.

As with any major transition period, while it comes with excitement for new beginnings it is critical to keep priorities aligned, goals set and move with an open-mind for potential changes and challenges.

I wish all the best for everyone in the New Year with lots of success, health and happiness.

RTC (Real-Time Communication) at scale

2020-05-02T19:49:29-04:00

I recently did an internship at Facebook where I was part of the RTC (Real-Time Communications) org, responsible for the products and infrastructure powering voice/video calling across Facebook’s family of apps (such as Messenger and Instagram). My team was responsible for the P2P and multiway calling protocols, including lower-level services written in C++ and platform integrating with partner teams in PHP.

This post will dedicate itself to discussing some of the protocols that go into audio and video calling.

It is not related to Facebook itself in any way. You can view the PDF version of this post here.

Executive Summary

Introduction
Analysis
- 2.1 Overview of Signaling and Media Streaming
- 2.2 History of Signaling and Media Streaming
- 2.3 ITU-T H.323 Protocols
- 2.4 Session Initiation Protocol (SIP)
- 2.5 Signal Protocol (formerly TextSecure Protocol)
- 2.6 WebRTC Framework
- 2.7 Real-Time Streaming Protocol (RTSP)
- 2.8 Real-Time Control Protocol (RTSP)
- 2.9 Real-Time Transport Protocol (RTP)
- 2.10 Audio Streaming Codecs
- 2.11 Video Streaming Codecs
- 2.12 Dynamic Adaptive Streaming over HTTP (DASH)
- 2.13 Multiway Calling Architectures
Case Studies
- 3.1 Skype’s P2P Signaling Protocol (2003)
Conclusion
- 4.1 Resulting architecture of RTC systems

Executive Summary

This post discusses and analyzes different protocols and technologies used in real-time communication systems by social media platforms. Real-time communication typically involves both a signaling and media stack. During signaling, actions are performed, such as ringing and hanging up, which often need to be relayed to users in an efficient manner. Similarly, media involves the encoding, relay and encryption of a stream of data, which includes audio and/or video. This post primarily focuses on a high-level overview of the different stacks, along with open-source protocols defined for use in this context. In addition, this post discusses different algorithms that are applied for media streaming, in the context of codecs and the perspective of performance.

The objective of this post it to raise familiarity with the challenges associated with real-time communications, and how different companies tackled problems related to it. In particular, this post focuses on the signaling and media streaming aspects, and not on underlying infrastructure details such as ensuring durability or reliability of hardware.

Introduction

The objective of real-time communication in the context of social media is to connect people from different avenues of the world together in a lifelike fashion. By allowing peer-to-peer or multiway communication through audio and/or video, users can connect in a personable way despite being geographically disperced. Billions of calls are performed everyday over the internet, including those from popular sites such as WhatsApp which support over 1 billion daily users (Mansoor Iqbal, 2020). During crisis and major world-events (such as COVID-19), real-time communication and social media can truly bring the world together in a safe manner.

Numerous technical challenges arise from designing scalable systems that handle users from different regions. First includes how to initiate the call and inform users that another user wishes to connect. Afterwards, there needs to be a direct connection established to allow for a reliable stream of data to be sent to the recipient(s). Moreover, there are variable network conditions that need to be accounted for such as low bandwith, unreliable connections or firewalls. In addition, there is the consideration of encrypting data sent over the network, and ensuring privacy of users by designing robust systems. This is just a small sample of the set of challenges faced by software engineers, architects and site reliability engineers in designing the world-class systems powering the large-scale social media websites of today.

Analysis

Overview of Signaling and Media Streaming

The high-level goal of real-time communication is to connect a party of people together such that they can communicate with low latency. To achieve this, two connections are established: signaling and media. Signaling is the process of exchange information between different parties, typically associated with events or actions. Media is the actual media being sent, which includes audio and/or video.

Figure 1: The simplest form of peer-to-peer communication.

Signaling is primarily used for session control, network data, media metadata and key material. Session control involves using messages to open, modify and close communication channels. Network data consists of any information for connecting the endpoints, such as obstructions in access. Media metadata is used to transmit metadata such as media types supported, codecs, along with encryption schemes used. Lastly, key material is used for sending encryption keys used to decode sensitive data sent over the network.

Media streams are used to send the data that users want to experience, such as audio and/or video. Audio and video media is typically encoded using different codecs and can be fine tuned depending on network conditions. Moreover, audio and video media is often susceptible to packet loss, as performance is more critical than losing individual packets of information. Various techniques have been researched and developed over the years to allow for more effective lossy/loseless compression, improved sound quality (echo cancellation, jitter avoidance, etc.) and remidiation of intermittent packet loss.

History of Signaling and Media Streaming

Signaling is not a modern concept - in fact, it was first introduced in the 1800’s as a major component of telephone systems. Electric pulses and audible tones are used to transmit information such as busy signalings, addressing, dial tones and to request service. The rotary dial in 1896, which was later formalized by 1910, introduced the first design of performing such requests which relayed electrical pulses in current flow (“Development of long-distance transmission”, n.d.). Over time, the standards incrementally envolved to take into consideration overseas transmission, newer transmission mediums such as coaxial cabling and applied techniques such as FDM (frequency division multiplexing) to best utilize existing infrastructure.

In contrast to telephone communication, real-time communication used by social media websites typically uses VoIP (Voice over Internet Protocol) versus PSTN (public switched telephone network). While general concepts remain consistent (such as signaling, connection setup, digitization of analog signals via quantization), data is transmitted over a packet-switched network versus circuit-switched network.

The first providers of VoIP mirrored solutions of the legacy telephone sys- tems. Second generation providers including Skype adopted large-scale closed networks providing free calling capability with the ability to tap into PSTN networks if preferred at cost. The latest generation leverages federated VoIP, which is a form of voice telephony using packets across autonomous domains without requiring switching centers or centralized virtual exchange points. An example would include Google Talk which uses ENUM (E.164 Electronic Number to URI Mapping standard), acting similarly to DNS record types where a telephone number would map to a URI or IP address.

VoIP calling is implemented using a combination of propriertary and open-source protocols. The foundation was formed in 1928 by Homer Dudley at Bells Labs creating the first electronic voice synthesizer (known as Vocoder). Later, ARPANET (Advanced Research Project Agency) built the first packet-switched network in 1969. By 1973 the first voice data packet was transmitted by MIT, followingly the first audio codec was approved by 1988 (G.722), and then finally by 1991 Autodesk released the first VoIP application (known as Netfone) to the public domain. Development increased substantially after that point, leading to Free World Dialup in 1994 and the first-for-profit VoIP application VocalTec in 1995 (Robert Pepper, 2014).

The H.323 system specification, instated in 1996, is recommended by the ITU Telecommunication Standardization Sector (ITU-T) as a stack of proto- cols used to provide audio/video communication on packet-switched networks (Margaret Rouse, n.d.). The standard addresses call signaling, control, mul- timedia transport and control, along with bandwith control for peer to peer and multiway conferences. As part of the H.323 family of telecommunication protocols includes H.225.0 RAS (Registration, Admission, Status), H.225.0 Calling Signaling, H.245 control protocol and Real-Time Transport Protocol (RTP) for delivering media.

SIP (Session Initiation Protocol) was originally standardized in 1999, and is designed to dictate signaling and call setup protocols for IP-based communications (Tien-Thinh Nguyen, Christian Bonnet, 2016). Unlike H.323, which is was standardized by the International Telecommunication Union (ITU), SIP was formalized by the Internet Engineering Task Force (IETF), hence distin- guishing its roots in the internet community rather than telecommunications industry. SIP works in conjunction with other protocols including SDP (Ses- sion Description Protocol) to negotiate and establish call state.

With respect to streaming media, MPEG-DASH (Dynamic Adaptive Streaming under HTTP, also known simply as DASH) was published in 2012 as the only adaptive bit-rate HTTP streaming solution recognized internation- ally for delivering variable bitrate content. This codec agnostic solution is used by YouTube, Netflix and more. Popular codecs used today for encod- ing video streams in real-time communication includes H.264 (MPEG-4 AVC) introduced in 2003 constituting the majority of tra c, and H.265 (MPEG-H HEVC) introduced in 2013.

ITU-T H.323 Protocols

H.323 is a standard that defines a series of protocols to provide audio/video communication sessions across packet-switched networks. The standard, created in 1996 by ITU-T (ITU Telecommunication Standardization Sector) addresses call control and signaling, multimedia control and transport, along with bandwith control for both peer-to-peer and multiway conferences (Margaret Rouse, n.d.). While originally created for use in LAN networks, it was quickly adopted for a variety of IP networks including WANs and the greater internet. The standard has since been iterated on, last updated in 2009, while remaining entirely backwards compatible since its first version. It resides on TCP port 1720.

Firstly, several definitions will be addressed.

Definition. 2.1. Terminal The fundamental element in a H.323 system representing a device users would typically use (such as a phone).

Definition. 2.2. Multipoint Control Unit (MCU) Device responsible to acting as a conference bridge and allows mixing both video/audio.

Definition. 2.3. Gateway Device responsible for enabling communication between H.323 networks and other network types (such as PSTN). An example of using gateways includes allowing enterprise IP phones to communicate with external users via PSTN.

Definition. 2.4. Gatekeeper An optional component across the H.323 network which provides a series of services to terminals and gateways. Examples include endpoint registration, admission control, and user authentication.

Gatekeepers use one of two signaling modes: direct and gatekeeper routed. When direct, endpoints use RAS protocol to learn the endpoint of the remote endpoint and a call is established directly with the remote device. In contrast, when using gatekeeper routed the gatekeeper is used as the intermediary and all signaling passes through it. Definition. 2.5. Zone The set of endpoints registered to a single Gatekeeper in a H.323 system.

After the address of the remote terminal is determined the initiating endpoint will leverage H.225.0 call signaling to establish communication with the remote terminal. Fast-connect procedures are defined in H.323 to establish calls using only 2-3 messages, reducing latency. Endpoints generally use RAS protocol to communite with a gatekeeper, while gatekeepers also use RAS protocol to connect with other gatekeepers.

A terminal would typically invoke a GRQ (Gatekeeper Request) message for discovering gatekeepers that are willing to accept messages. After, applicable gatekeepers would confirm (GCF) and the terminal node would pick its most desirable gatekeeper.

Upon sending a call initiation signal, the terminal node would perform an admission request (ARQ) to its preferred gatekeeper. In response, the resolves address is returned in the form of an admission confirm message (ACF). Now, the initiating terminal node will send the ARQ to the remote endpoint. The remote endpoint will then likewise send an ARQ and receive an ACF from its preferred gatekeeper. This step is required for ensuring the device is properly authenticated to call the other recipient, and network conditions are capable of sustaining a call.

Figure 2: Signaling between two endpoints with gatekeepers using H.225.0.

After the call was initiated, the endpoints may further invoke H.245 call control signaling to provide more control. H.245 is capable of sending information required such as encryption, jitter management and preferences (disabling audio and/or video). However, one disadvantage of H.245 is the lengthy four-way protocol handshake required when opening logical channels of a conference. This was alleviated using fastStart in a H.225.0 message (“H.323 Fast Start”, 2015). Later, H.460.6 introduced Extended Fast Connect Feature, providing a one-way handshake.

Session Initiation Protocol (SIP)

Session Initiation Protocol, standardized in 1999, is a signaling protocol used for the initiation, maintenance and termination of real-time voice/video applications (Tien-Thinh Nguyen, Christian Bonnet, 2016). The protocol is text-based, and borrows its structure from Simple Mail Transfer Protocol (SMTP) and Hypertext Transfer Protocol (HTTP). SIP is used alongside Session Description Protocol (SDP) as payload containing media metadata, and was designed to be independent of the transport layer. Therefore, SIP can be transfered over Transmission Control Protocol (TCP), User Datagram Protocol (UDP) or Stream Control Transmission Protocol (SCTP). In addition, SIP transmission is accompanied by encryption using Transport Layer Security (TLS). In contrast with H.323, SIP has been standardized by the IETF rather than the ITU, hence has a stronger connection to the internet community than telecommunications.

First, several definitions will be addressed.

Definition. 2.6. User Agent Client (UAC) User Agent capable of receiving responses and sending requests.

Definition. 2.7. User Agent Server (UAS) User Agent capable of receiving requests and sending responses.

Definition. 2.8. User Agent Network endpoint that receives SIP messages and is capable of managing SIP sessions. Unlike other protocols, SIP requires all user agents to implement both client and server roles, which includes ability to handle both requests and responses.

Definition. 2.9. Proxy Server Network endpoint that receives SIP messages and is capable of managing SIP sessions. Unlike other protocols, SIP requires all user agents to implement both client and server roles, which includes ability to handle both requests and responses.

Definition. 2.10. Registrar SIP endpoint that accepts REGISTER requests, and records the address from the requesting user agent. Registrars are often used to locate other user agents.

SIP can be used for both peer-to-peer and multiway conferences. Typically, both user agents will register with the registrar by providing REGISTER requests along with any authentication. Afterwards, the UAC will signal requests (such as an INVITE) to the UAS.

A selection of common requests/responses is included in Table 1 and 2.

Request Name	Description
REGISTER	Registers the caller’s URI to the registar, used by “To” field
ACK	Confirms user agent receiving final response to INVITE
BYE	Signal that the call should be terminated
INVITE	Initiate dialog from UAC to UAS to begin call
CANCEL	Cancel any pending requests (such as pre-existing rings)
REFER	Ask recipient to takeover existing call (and issue request)
INFO	Send any conference metadata while not modifying session state
MESSAGE	Deliver a text message, typically used by IM clients
PUBLISH	Deliver an event to a notification server
OPTIONS	Retrieve the capabilities of an endpoint/user agent

Table 1: Session Initiation Protocol Requests

Response Code	Description
1xx	Request was valid and is being processed (asynchronously)
2xx	Successful completion (for INVITE, it indicates an accept)
3xx	Call redirection required (INVITE to another user agent)
4xx	Invalid request (potentialy due to incorrect request syntax)
5xx	Server failure (potentially internal errors)
6xx	Global failure (not limited to one server, can be invalid destination)

Table 2: Session Initiation Protocol Responses

Lastly, an extension of SIP is the Session Initiation Protocol for Instant Messaging and Presence Leveraging Extensions (SIMPLE), which can be used for features such as the ... typing indicator, and adds more specialized request/response headers for instant messaging. The first line of requests contains the method along with the request URI.

On the other hand, the first line of responses is the response code.

Signal Protocol

The Signal Protocol, started in 2013 by Open Whisper Systems, is a non-federated cryptographic protocol allowing for end-to-end encryption of voice/video calls and instant messaging conversations. Originally it was used in the popular messaging app Signal, but later was adopted by other apps including WhatsApp and Messenger/Skype in their optional private conversation modes.

By implementing end-to-end encryption, the contents of messages and calling signals/media are only available to the sender and recipient(s), and not the intermittent servers and devices. First, a set of long-term identity key pairs, medium-term signed prekey pair and several ephemeral prekey pairs are gen- erated locally and stored securely. After, all the public keys and registration ids are stored in a “key bundle” and registered with the Key Distribution Center. For the sender to send messages, the sender has to be able to get the registration ID and public keys from the receiver, which can be found in the publically accessible key bundle.

To start a session, the sender uses their identity and medium-term private keys along with the recipient’s set of private keys to determine the master shared secret. The recipient can then receive the master shared secret, decipher and validate it. Afterwards, the two users can send messages to each other.

While the session is alive, the sender uses the master shared key and the recipient’s ephemeral keys to create a root key, chain key and message chain. This results in a new set of single-use epehemeral keys to encrypt/decrypt future messages. In total, the protocol combines the Double Ratchet algorithm, prekeys, an extended triple Elliptic-curve Diffie-Hellman (X3DH) key agreement protocol, along with using Curve25519, AES-256 and HMAC-SHA256 as cryptographic primitives (Ksenia Kozhukhovskaya, 2017).

X3DH is used for generating all the required keys for the sender/receiver to communicate, including the shared secret key between the two callers, by registering identity and prekeys to a server. Using this architecture, the caller can retrieve the “prekey bundle” of the callee even when the callee is online.

Double Ratchet Algorithm is used to provide end-to-end encryption based on the shared secret key retrieved by X3DH. From the shared secret key produced, a “root key” and “sending chain key” are generated. Using a key derivation chain (KDF) from the sending chain key, each subsequent message has a differ- ent epehemeral key by advancing down the chain, and it makes it impossible to decrypt out-of-order messages. In short, when encrypting messages, the sender always forwards the sending chain by one, and generates a new sending chain key and messaging encryption key.

Finally, AES-256 and HMAC-SHA256 are both 256-bit length encryption functions/block ciphers to protect and encrypt sensitive data. Using the master private keys that are shared across caller/callee, the schemes are used to ensure that the data is essentially non-recoverable without the proper credentials.

WebRTC Framework

WebRTC (Web Real-Time Communication) is an open-source project providing mobile applications and web-browsers with RTC capabilities via simple APIs. The mission of WebRTC is to “enable rich, high-quality RTC applications to be developed for the browser, mobile platforms, and IoT devices, and allow them all to communicate via a common set of protocols”.

First, several definitions will be addressed.

Definition. 2.11. Session Description Protocol (SDP) Format used in describing and negotiating a session’s profile (which includes properties and parameters such as media types and encryption keys). SDP can use attributes extending the protocol’s capabilitiesas key/value pairs, and is otherwise a text- based format with one field per line.

Definition. 2.12. NAT (Network Address Translation) Process for translating local IP addresses to public IP addresses, often used for security reasons and conserves legally registered IP addresses (due to limitations with IPv4 bit count).

NATs typically work in four ways: full cone (internal address maps to external address and any external host can send requests), address-restricted cone (any host can send to an externally mapped host/port if and only if that host/port previously sent to the host), port-restricted cone (similar to address-restricted cone, except not only did the receiver need to have previously sent to the same host, but also the matching sender port), and lastly symmetric (where a new mapping is used for each request).

Definition. 2.13. STUN (Session Traversal Utilities for NAT) Lightweight and simple service which provides the public IP address of its caller, used to reply back its IP address to the original caller to establish an IP address; typically used in non-symmetric NATs (full-cone, address-restricted, port-restricted).

Definition. 2.14. TURN (Traversal using Relays around NAT) Computationally expensive relay for sending messages/media to a destination, typically used in symmetric NATs where the public IP cannot be discoverable and instead requires a statically defined external service to send information to the requesting caller.

Definition. 2.15. ICE (Interactive Connectivity Establishment) Standard used for performing NAT (Network Address Translation) traversals. ICE deals with returning candidate agent addresses (local, reflexive such as STUN and relayed such as TURN).

Definition. 2.16. Trickle ICE Optimization of ICE specification providing parallelized connectivity checks across the candidate addresses, reducing the overall initiation time.

There are several core components of the WebRTC framework that is exposed through the JavaScript APIs. Some examples include “getUserMedia” that is used to acquire the audio/video media. In addition, “RTCPeerConnection” is used for performing signal processing, security, peer-to-peer communication and bandwith management. Another core capability is the “RTCDataChannel” for allowing bidirectional communication among peers for sending media using a system based on WebSockets (MDN Contributors, 2020).

The first task in establishing a WebRTC connection is to use a signaling server for resolving the connection. The role of the signaling server is to act as an intermediary and allow the peers to establish a connection with minimizing pri- vate information exposure. WebRTC does not mandate any specific transport mechanism. Instead, WebRTC leverages SDP payloads and ICE to determine candidates for establishing the connection.

For exchanging media WebRTC uses SDP to execute the offer and answer mechanism across peers. Unfortunately, firewalls and NATs are often used in the real-world to both protect private IPs and limit the number of registered IP addreses due to limited IPv4 addressing. However, NAT mappings would occur at the network layer, and hence will change the TCP/UDP packet headers but leave SDP payloads unchanged hence leaving them unaware of handling external NAT IP addresses and port restrictions.

Figure 3: Example of simple signaling and media transfer without NATs.

Depending on the type of NAT (full cone, address-restricted/port-restricted cone or symmetric), a different configuration of services is required. STUN is used in the case of asymmetric NATs, where it is simply invoked to determine the public IP address and then that can be used for media transfer. This is a lightweight solution and is common in practice. Unfortunately, for symmetric cones a relay solution is required where TURN servers will be used as the intermediary endpoints. The disadvantage of this approach is that TURN servers are expensive, add additional latency and will be another source of failure among the media streaming path.

Figure 4: Example of signaling and media transfer between coned/reflexive NATs.

Popular users of the WebRTC users includes Facebook with over 300 million monthly active users, Discord and Skype (Chad Hart, 2017).

Real-Time Streaming Protocol (RTSP)

Real-Time Streaming Protocol (RTSP), standardized in 1998 by the IETF, is a network control protocol for controlling streaming media servers (Ivn Santos- Gonzlez, Alexandra Rivero-Garca, Jezabel Molina-Gil, Pino Caballero-Gil, 2017). RTSP is not responsible for the streamed data itself, which is often handled by the Real-time Transport Protocol (RTP).

Instead, it facilitates capabilities such as play and pause. The protocol is similar to HTTP, with several exceptions such as RTSP being stateful and resides on both TCP/UDP ports 554 (although UDP is rarely ever used).

RTSP supports several types of requests, including several overlapping with HTTP. “OPTIONS”, similar to HTTP, returns the accepted request types by the server. “DESCRIBE” requests are used for obtaining SDP presentation descriptions, which contain metadta such as the list of media streams that are controlled by the aggregate URL, bitrate, mime type, etc. Lastly, there are “PLAY”, “PAUSE” and “TEARDOWN” requests.

Popular client implementations of this protocol include VLC media player, Skype, Spotify, QuickTime and Windows Media Player. In addition, YouTube uses RTSP as an available streaming option when the mobile HTTPS site is viewed on desktop.

Real-Time Control Protocol (RTCP)

Real-Time Control Protocol (RTCP) is often used in conjunction with RTP to provide out-of-band (signaling running on a dedicated channel away from media channels in PTSN) control information and statistics for an RTP session. RTCP provides feedback on the quality of service including roundtrip delay time, packet delay variation, packet loss and packet counts. This allows for adaptive capabilities such as Dynamic Adaptive Streaming over HTTP (DASH) to be implemented (“RTP Control Protocol (RTCP)”, 2017). RTP is generally executed on an even-numbered UDP port, and RTCP on the next odd-numbered port. While RTCP does not provide any encryption/authentication methods, Secure Real-time Transport Protocol (SRTP) provides such capabilities.

Figure 5: Example of signaling between two peers in WebRTC.

Figure 6: Example of ICE negotiation in WebRTC.

Web-Browser	Supported Versions
Microsoft Edge	12+
Google Chrome (Desktop)	28+
Mozilla Firefox	22+
Opera	18+
Safari	11+
Google Chrome (Android)	28+
Mozilla Firefox (Android)	24+
Opera (Mobile)	12+
iOS MobileSafari/WebKit	iOS11+

Table 3: Browsers with WebRTC Support

RTCP provides three functionalities. First, RTCP provides statistics on qual- ity of service during media distribution. This can be used, as mentioned earlier, for the implementation of DASH alorithms and paging Engineers to look into potential network disruptions. Secondly, RTCP provides Canonical Name Records (CNAME) to conference participants and allows for effective third-party monitoring. Lastly, RTCP is often used for reaching all conference participants, as RTP only transmits via media source. RTCP reporting is applied to all participants of a conference, and is randomized in reporting time to avoid unintended synchronization of reporting (typically intervals around five seconds are used). As a best practice to avoid network congestion, RTCP bandwith usage should not be higher than 5% of overall session bandwith to prevent causing degraded performance.

Several types of messages are supported by RTCP, and can be extended further to include custom packets. Sender report (SR) is periodically sent by active senders in conferences to report reception and transmission statistics. Receiver report (RR) is for receivers of RTP packets, and likewise sends quality reports to senders. Source description (SDES) provides CNAMEs to session participants for use in third-party monitoring solutions. Goodbye (BYE) is used to shut down a tream of data.

Real-Time Transport Protocol (RTP)

Real-Time Transport Protocol (RTP), standardized by the IETF in 1996, is a networking protocol for audio and video delivery over IP networks. It is often used in conjunction with SIP and RTCP to create and ensure quality transmission of media. RTP is designed for real-time transfer of media streams and provides capabilities such as detection of packet loss, out-of-order delivery, jitter compensation and IP multicast support. RTP is based on application- layer framing, where the protocol functions are implemented in the application rather than operating system.

RTP supports a range of multimedia formats and is extensible by design. By providing profiles and one or more payload formats for each class of application (such as audio or video), RTP supports definining mapping codecs to payload format codes inside profiles, which then describes the transport method for the encoded data. For example, Secure Real-Time Transport Protocol (SRTP) is an RTP profile for providing cryptographic services for transferring payload data. RTP senders would capture the media, encode it, then transmits it as an RTP packet with the appropriate timestamps and increasing sequence numbers. RTP receivers then detect missing packets and may reorder packets.

Afterwards, the stream can be decoded and presented to the user. Packet headers for RTP support various mandatory fields, as shown in the figure below. Version is used to indicate the current version of the protocol used (2 bits, where the current version is 102). P (Padding, 1 bit) may indicate extra padded bytes at the end of a header, typically used due to size requirements imposed by encryption algorithms. X (Extension, 1 bit) is used to indicate presence of an extension header between header and the payload. CC (CSRC count, 4 bits) indicates the number of CSRC identifiers fllowing SSRC. Sequence numbers are incremented for each RTP data packet sent, used to implement out-of-order delivery and detect lost packets. SSRC (synchronization source identifier, 32 bits) is used to identify the source of the stream, while CSRC (contributing source ids, 32 bits each) are used to enumerate contributing sources to the stream. Lastly, the optional header extension would include a 16 bit profile-specific identifier and a 16 bit length specifier, with the custom extension header data following.

Figure 8: Example of an RTP packet header.

Audio Streaming Codecs

The process of encoding and decoding audio data is handled by an audio codec (coder/decoder). In real-time communication, the balance between having high-quality audio and low-bandwith requirements is a power struggle that often requires compromises from either side. The two general categories of factors affecting encoded audio is the codec chosen and the details about the source audio’s contents and format.

Different codecs provide a variety of parameters to tune allowing for vari- able bit rate, audio frequency bandwith and many other custom fields. Audio codecs are based on advanced numerical computational algorithms and can provide either lossless or lossy compression. AAC (Advanced Audio Coding) is a common codec defined by the MPEG-4 (H.264) standard, used by Blue-ray disks, HDTV, and songs purchased from iTunes (MDN contributors, 2019). However, the format is protected by numerous patents making distribution less predictable. On the other hand, G.722 codec is built with voice compression in mind, is low-latency and uses Adaptive Differential Pulse Code Modulation (ADPCM) to reduce recording size. G.722 is mandated by the WebRTC specification and is typically used for WebRTC connections. Lastly, MP3 (MPEG-1 Audio Layer III) is one of the most common codecs, where MPEG-1 MP3 is generally used for music and MPEG-2 MP3 is used for simpler sounds and requires less space. Moreover, MP3 patents expired as of 2017 in the United States making it supported by all popular web-browsers.

The WebRTC API does not mandate if a particular can be used in a track, but instead requires support VP8 and H.264’s constrained baseline profiles for video and G.711 PCM (A-law) and G.711 PCM (u-Law) audio codecs for Chrome, Firefox and Safari browsers.

Video Streaming Codecs

Video encoding is the process of turning raw video into a digital format to be viewed on different devices. Similar to audio codecs, video codecs vary in behavior and specialization depending on use-case, and are often either lossless or lossy in nature. Encoded videos are wrapped into a “video container” (such a .mp4, mov), which contains the video codec, audio codec and associated metadata. The key difference between a container format and a codec is that the codec is at the source and playback to compress and decompress respectively, while the container can be used to determine which programs accept the stream and holds the different components (audio, video, closed captioning) together.

The most common encoding for streaming is MPEG-4 H.264/AVC (Advanced Video Coding), developed by the International Telecommunications Union (ITU) and International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) Moving Picture Experts Group. H.264/AVC has capabilities of being packaged into a variety of container types including .mov, .mp4, .3GP and more.

MPEG-5 H.265/HVEC is a newer codec, which has improved compression efficiency and supports 8k resolution. However, the market-share is only around 10% due to royalty complications (Traci Ruether, 2019). To address this, AV1 was created in a partnership with several leading tech companies (Google, Mi- crosoft, Amazon, Netflix, and more) to create a performant royalty-free codec. Google created the VP9 codec as a royalty-free and more performant version of HVEC, however it is not supported on Apple devices and is often considered to be “AV0”.

Dynamic Adapative Streaming over HTTP (DASH)

Dynamic Adaptive Streaming over HTTP (MPEG-DASH, otherwise known simply as DASH) was standardized in 2012 to provide adaptive bitrate streaming for media over HTTP web servers. DASH works by taking content and breaking it down into smaller HTTP-based file segments, where each segment contains a small interval of content. Each individual segment is then broken down into different bit rates. While the media is streamed, the client applies a bit rate adaption (ABR) algorithm that selects the appropriate segment such that the content can be downloaded in time without causing rebuffering/stalls and maximizes quality.

DASH uses TCP as the chosen transport protocol, and remains agnostic to codecs (meaning it supports various formats including H.264, H.265, VP9), ABR logic and the underlying application layer protocol. Alongside the individual segments containing intervals of streamed data, DASH uses a media presentation description (MPD) to describe segment information (URL, timing, bit rates, etc.), and is presented using a variety of data-structures such as timelines or lists. While there is no restrictions on the type of media data, the specification provides recommendations on using two types of containers: MPEG-2 Transport stream or an ISO base media file format (such as MP4).

Adoption is widespread for DASH, including support by YouTube and Netflix and VLC. While not directly supported in HTML5, there are open-source JavaScript implementations for DASH adding the functionality. Moreover, when combined with WebGL, HTML5-based DASH allows for streaming of 360 degrees content. Alongside developer and product support, content distribution networks (CDN) support for DASH is vast, including Akamai, Amazon CloudFront, CloudFlare and Azure Media Services.

HTTP Live Streaming (HLS), released in 2009, is an alternative HTTP-based adaptive bitrate streaming protocol developed by Apple (Max Wilbert, 2020). Similar to DASH, HLS breaks the stream into a sequence of smaller HTTP-based file downloads in a variety of different bitrates. After, the list of available streams encoded at different bitrates is sent as an extended M3U playlist to the client. Unlike RTP, HLS allows for traversing firewalls and NATs that would otherwise allow regular HTTP tra c to travel through. Later versions of the protocol also introduced subtitle support. The architecture of HLS is comprised of three parts: server, distributor and client. The server is responsible for preparing the video for delivery, encoding encoding the video files in H.264 format in variable bitrates as MP3, AAC, AC-3 or EC-3, and then encapsulated in MPEG-2 Transport Stream. After, the segmenter will divide the MPEG-2 TS file into equally sized segments, and then create the index file for fragmented files as an m3u8 file. The distributor will then act as a regular HTTP web server, and deliver the required m3u8 playlist file and ts segment files required to stream the content. Lastly, the client will retrieve the m3u8 file containing the index of segments, and then retrieve the necessary segments from the distributor.

Multiway Calling Architecture

WebRTC media streaming natively supports communication across two different peers, but often real-time communication applications support multiway calling involving numerous peers. Multiway conferences for voice and video can be supported using three architectures: mesh, mixing and routing (Tsahi Levent-Levi, 2019). In addition, signaling often uses an intermediary centralized server acting as an anonymous “peer”, and relays signaling using ICE candidates (such as STUN or TURN).

Mesh routing involves an n^2 amount of linkages, where n is the number of peers, which is not scalable to many users and requires substantial amounts of bandwith. Mixing leverages an MCU (Multipoint Conferencing Unit) which acts as a centralized point where multiple streams from individual peers are combined into a single unified stream. However, despite the simplicity in design, it comes without flexibility such as client-specific processing because the streams are combined at the relay source.

Routing leverages an SFU (Selective Forwarding Unit), which instead acts as the router of the media (Tsahi Levent-Levi, 2019). In contrast to mixing, routing will send individual streams directly to other peers allowing for more client-sided flexibility in terms of processing. Three approaches are applied for routing: multi-unicast, simulcast and SVC. Multi-unicast is the trivial approach, where users would send streams to the SFU and the SFU would decide where to route, not performing any bit rate adaption. On the other

Figure 8: Example of multiway mesh streams.

Figure 9: Example of multiway mixing streams.

Figure 10: Example of multiway routing streams.

hand, simulcast will accept multiple streams of varying qualities and then will send the supported ones (based on network conditions and device capabilities) to the other peers.

Lastly, SVC (scalable video coding) uses a similar methodology as simulcast, but instead of sending disjoint streams of varying bitrates, SVC sends a layered stream of increasing quality where particular layers can be “peeled” off the top to reduce quality. This is an enhancement over simulcast because it reduces computation time and allows error corrections to occur only at base levels; it was introduced to WebRTC in the VP9 video codec.

Case Studies

Skype’s P2P Signaling Protcol (2003)

Skype is a VoIP client developed by Microsoft allowing users to place audio and video calls over the internet. Skype uses an overlayed peer-to-peer network, similar to its file sharing predecessor Kazaa (Salman A. Baset, Henning G. Schulzrinne, 2004). There are two types of nodes in the architecture: ordinary hosts and super nodes (SN). An ordinary node is any Skype client that is used for issuing voice and video calls. Super nodes are also Skype clients, however they are promoted to SN once it is identified that they have a public IP address, performant hardware (RAM, CPU) and network bandwith. In addition, the login server is a critical piece of Skype infrastructure as it handles the login credentials of users and in later Skype versions the friends lists of users. Moreover, SkypeOut and SkypeIn are servers used to bridge VoIP with PSTN, however it is not used in pure VoIP calls.

Every Skype node uses a variant of the STUN/TURN protocol to identify NAT and firewall restrictions of users. When the SC is loading, it first sends an HTTP request to determine if there should be a software update. After, a connection to a SN is established. A critical component of Skype clients is the host cache (HC), which is built and refreshed regularly to contain a list of SN capped at a length of 200. If none of the SN inside the HC are reachable, Skype resorts to establishing a TCP connection to a bootstrapped list of 8 hardcoded SN addresses - if that does not work either then Skype fails to login. After SC is connect to a SN, the client applies the username/password to authenticate with the Skype login server.

Skype user search leverages a Global Index (GI) technology (Salman A. Baset, Henning G. Schulzrinne, 2004). First, the SC will ask the SN over UDP if it knows any users matching the regular expression. If an SN does not know, then it will provide 8 addresses of SN nodes over TCP to the SC to further query. This process repeats with 16, 32, and exponentially many more nodes. After an abritrary cutoff, if none are found, the login server is requested (which is always invoked in the case of no matching usernames). Repeated queries are cached locally at the client.

The Skype protocol has no silence suppression, meaning it still sends UDP packets containing null noise even when muted. There are two benefits to this design choice, namely preventing reapplying UDP bindings and in the case of TCP being used it prevents drops in the TCP congestion window which would reduce preliminary RTT time until it ramps back up. The codec of choice used by Skype is iCodec, with a minimum bandwith requirement of 2Kb/second (Salman A. Baset, Henning G. Schulzrinne, 2004). Lastly, it was observed that in conferences, the media is not fully meshed, meaning the most powerful machine is elected to be the host and collect and later distribute the streams to the remaining hosts, as show in the figure below.

Figure 11: Example conference signaling without full mesh.

Conclusions

Resulting architecture of RTC systems

This post provides a high-level overview of different protcols, standards and codecs used to implement scalable real-time communication services such as video calling. Various protocols and technologies are involved in the signaling, encoding, and delivery of media over networks. Over time, numerous opti- mizations have been applied to reduce the number of handshakes required to establish connections, save bandwith and improve call quality. By applying the techniques applied in this post, it becomes possible to get a high-level view of designing a large-scale system supporting numerous concurrent users across ranging network setups such as private IP addresses guarded by fire- walls. In the future, frameworks such as WebRTC will become more mature, codecs such as H.265 will have widespread adoption and protocols will be added making it easier than ever to keep the world connected and together by using real-time communication.

References

[1] Mansoor Iqbal (2020). WhatsApp Revenue and Usage Statistics. Business of Apps (2020). Retrieved April 12, 2020, from https://www.businessofapps.com/data/whatsapp-statistics/

[2] (n.d.) Development of long-distance transmission. ENCYCLOPDIA BRITANNICA. Retrieved April 12, 2020 from https://www.britannica.com/technology/telephone/Transmission

[3]Robert Pepper (2014). The History of VoIP and Inter- net Telephones GetVoIP. Retrieved April 12, 2020, from https://getvoip.com/blog/2014/01/27/history-of-voip-and-internet- telephones/

[4] Jan Ozer (2011). What Is MPEG DASH? Streaming Media. Retrieved April 12, 2020, from https://www.streamingmedia.com/Articles/ReadArticle.aspx?ArticleID=79041

[5] Margaret Rouse (n.d.). H.323 Search Networking. Retrieved April 12, 2020, from https://searchnetworking.techtarget.com/definition/H323

[6] Tien-Thinh Nguyen, Christian Bonnet (2016). IP Mobility Management for Future Public Safety Networks Wireless Public Safety Networks 2. Retrieved April 12, 2020, from https://www.sciencedirect.com/topics/computer-science/session- initiation-protocol

[7] (2015). H.323 Fast Start. Dialogic.. Retrieved April 12, 2020, from https://www.dialogic.com/webhelp/BorderNet2020/1.1.0/WebHelp/h323 faststart.htm 31

[8] Ksenia Kozhukhovskaya (2017). Demystifying the Signal Protocol for End-to-End Encryption (E2EE) Cloudboost. Retrieved April 12, 2020, from https://blog.cloudboost.io/demystifying-the-signal-protocol-for-end- to-end-encryption-e2ee-3e31830c456f

[9] MDN Contributors (2020). WebRTC API. Mozilla Developer Network. Retrieved April 12, 2020, from https://developer.mozilla.org/en- US/docs/Web/API/WebRTC API

[10] Chad Hart (2017). WebRTC: One of 2016’s Biggest Technologies No One Has Heard Of Web RTC World. Retrieved April 12, 2020, from http://www.webrtcworld.com/topics/webrtc-world/articles/428444- webrtc-one-2016s-biggest-technologies-no-one-has.htm

[11] Ivn Santos-Gonzlez, Alexandra Rivero-Garca, Jezabel Molina-Gil, Pino Caballero-Gil (2017). Implementation and Analysis of Real-Time Streaming Protocols US National Library of Medicine. Retrieved April 12, 2020, from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5424723/

[12] (2017). RTP Control Protocol (RTCP) SDX Central. Retrieved April 12, 2020, from https://www.sdxcentral.com/resources/glossary/rtp-control-protocol-rtcp/

[13] MDN contributors (2019). Web audio codec guide Mozilla Developer Network. Retrieved April 12, 2020, from https://developer.mozilla.org/en- US/docs/Web/Media/Formats/Audio codecs

[14] Traci Ruether (2019). Video Codecs and Encoding: Everything You Should Know (Update) Wowza Media Streams. Retrieved April 12, 2020, from https://www.wowza.com/blog/video-codecs-encoding 32

[15] Max Wilbert (2020). What is HLS streaming and when should you use it? dacast. Retrieved April 12, 2020, from https://www.dacast.com/blog/hls-streaming-protocol/

[16] Tsahi Levent-Levi (2019). WebRTC Multiparty Architectures BlogGeek. Retrieved April 12, 2020, from https://bloggeek.me/webrtc-multiparty- architectures/

[17] Salman A. Baset and Henning G. Schulzrinne (2004). An Analysis of the Skype Peer-to-Peer Internet Telephony Protocol Columbia University. Retrieved April 12, 2020, from http://www1.cs.columbia.edu/ salman/publications/skype1 4.pdf

Living in Seattle for Four Months

2020-03-26T19:49:29-04:00

I am fortunate to have spent three (not four) months in Seattle, Washington at Facebook! Here is an overview of some activities and sightseeing accomplished during the term.

Overall, the Seattle winter weather didn’t really play in the favor of outdoor activities (fact: January had 28 days of rain), but nonetheless it was a fun term with friends and nice co-workers.

Why end early?

The Coronavirus (COVID-19) essentially sprung from being ever so distant in Wuhan, China but eventually landed at the doorsteps in Washington. After border restrictions were put in place, the University of Waterloo, Canadian Government and Visa Sponsorer (Cultural Vistas) all recommended to return.

While I originally refused (as I wanted to spend more time at Facebook), eventually CECA (UW’s co-op department) told Facebook to make co-ops return. In retrospect, it was the right decision for safety.

Living Situation

The corporate housing (624 Yale) was quite good - I’d place it between Bloomberg (92y) and Citadel (Block 37 Marquee) in terms of desirability. Fully furnished 2 bedroom 2 bathroom in South Lake Union made it easy to get to Facebook Dexter, and it was in a safe and quiet neighborhood. The bathrooms and kitchen were fully equipt with just about anything you’d need.

The bi-weekly cleaners were also a plus, in addition to in-suite laundary. The building’s gym was quite poor compared to Citadel’s rooftop gym, or even Bloomberg’s 92y gym. There weren’t any other amenities offered in the building, such as a swimming pool, and the lobby felt small. In addition, there was no 24/7 reception. However, it was substantially quieter than both Citadel and Bloomberg’s, so I enjoyed it.

Seattle’s already dark and gloomy, the lack of lighting was a major issue during the winter rainy months.

SWAG

Of course, not an accomplishment or venture but the most important part of any internship; during my internship at Facebook I was on the Messenger RTC (Real-Time Communications) organization, more specifically focusing on call signaling. Not pictured here is a free Facebook Portal Mini.

Mount Si

On the second weekend, I ventured out and hiked up Mount Si. I wasn’t entirely certain what to expect, but it turned out to be a fairly involved climb. The route can be seen below.

Coming in at around 1000m elevation gain (3300ft) over 14km roundtrip (8.7 miles), the route was certainly more involved than the Muir Woods trail in San Francisco several terms ago. The biggest difficulty wasn’t at all the actual route, but the lack of preparation on my part.

More specifically, I underestimated the amount of snow there was and I grossly missed on equipment such as spikes and poles. On several occasions I slipped on the ascent/descent, but slow and steady went ok in the end… in the future: gloves, spikes and poles.

Mount Squak

Another smaller hike followed, which wasn’t anywhere near as challenging and the view was just fog. Nonetheless, some statistics for that hike here.

Mount Snoqualmie

“Work from home Wednesdays”, aka paid ski trip for the entire Bellevue office.

Vancouver

Not the best weather, nor went up Whistler mountain, but nonetheless touring Stanley park was enjoyable.

Ride to Kenmore

One of my friends from high-school had just started full-time at Facebook Seattle and lent me his hybrid Giant bike. The Burke Gilman trail was particularily pleasant to ride on.

Living in Chicago for Four Months

2019-07-16T19:49:29-04:00

I am fortunate to have spent an internship at Citadel Securities in Chicago, Illinois.

Citadel Securities is the largest market maker in the United States responsible for executing over 20% of US equity trades, including over 39.5% of retail shares traded. You can watch this video outlining high frequency trading (plus you get to see my co-workers, it was filmed behind my desk!).

The provided corporate housing is a 2 bedroom + 2 bath luxury condo suite shared with 1 roommate being around 5 minutes walk from the office in the heart of the loop.

This has been by far the best corporate housing so far, we even had a weekly cleaner and lots of amenities including swimming pool, rooftop terrace/gym, billiards, and more!

The views from the corporate housing were great too.

This is the view of Chicago from the Planetarium.

The following are pictures of River North.

View of Chicago from the Botanical Garden.

John Hancock building on a foggy day.

Views from John Hancock Tower 96th floor

Ken Griffin, the founder of Citadel, paid over $300 million for this painting.

Citadel HQ (outside)

Chicago Botanical Garden

Lastly … 3oz Wagyu Kobe Beef :)

Horatiu’s Blog

Visiting Japan (東京マラソンを走る)

Ready for First Marathon!?

Training Strategy

Pre-Tokyo Marathon Plan

Race Day Strategy & Execution

Lessons Learned & What Worked

Post-Race Touristy Adventures

Retrospective

A week cycling in Girona and Catalonia

Training Leading up to Girona

Trip Strategy

Day #1: Els Àngels

Day #2: Mare de Déu del Mont

Day #3: Rest Day Run

Day #4: Tossa de Mar (Coastal Loop)

Day #5: Sant Hilari Sacalm and Susqueda Dam

Day #6: Rocacorba

Girona & Barcelona

Retrospective

YouTube Descents

Visiting Great Smoky Mountains National Park

Mount LeConte

Mount Cammerer

Cades Cove

Cligman’s Dome

Forbidden Carvens

Ziplining

Retrospective

A week cycling in Mallorca and Serra de Tramuntana

Getting sick right before the trip

Trip Plan

Day #1: Cap de Formentor

Day #2: Ermita la Victoria

Day #3: Andratx to Port de Pollenca

Day #4: Local Villages

Day #5: Rest Day, Sicker

Day #6: Skipped Final Ride

Retrospective

YouTube Descents

A week cycling in Bormio and the Alps

Day 1: Lago di Cancano

Day 2: Stelvio Pass + Umbrail Pass

Day 3: Sondalo

Day 4: Bormio 2000

Day 5: Passo di Gavia

Recap & Forward Look

YouTube Descents

Day 2 Descents (Stelvio + Umbrail Pass)

Day 5 Descent (Gavia Pass)

Visiting Grand Canyon, Death Valley, and Zion National Parks

Grand Canyon

Las Vegas

Death Valley

Zion

Conclusion

Living in Toronto for Eight Months (COVID-19 Edition)

What happened?

April - May 2020

May - August 2020

Google Internship

Road Biking + Tennis

YouTube (Learn in 5 Minutes Series)

September - December 2020

Forward Looking at 2021

RTC (Real-Time Communication) at scale

Table of Contents

Executive Summary

Introduction

Analysis

Overview of Signaling and Media Streaming

History of Signaling and Media Streaming

ITU-T H.323 Protocols

Session Initiation Protocol (SIP)

Signal Protocol

WebRTC Framework

Real-Time Streaming Protocol (RTSP)

Real-Time Control Protocol (RTCP)

Real-Time Transport Protocol (RTP)

Audio Streaming Codecs