Author: Dian Barber
FM Issue: Leadership and Operations 2015
The rising data demands of venue and special event attendees and the competition between facilities directly influence the decisions of managers to add costly improvements to their network infrastructures. Whether the work involves additions, renovations, or upgrades, the core imperative remains the same: Provide excellent customer experience at a time when the customer expects to connect—and stay connected—to a venue’s Wi-Fi system. Offering this capability is no longer a luxury; it’s a necessity.
And for the changing demographics of sports fans as well as attendees of other events, the expectations of connectivity are not only linked to professional requirements, but also to the changing mores of an ever-more connected society: They want to share their experiences with friends, family, and the world.
Joe Inzerillo, executive vice president and CTO of Major League Baseball’s Advanced Media (MLBAM), considers high-quality, consistent, robust, and pervasive bandwidth for fans as “essential for the digital lifestyle—just as important as making sure the toilets flush or the beer pours.”
Digital technomadism has become the new Archaic Revival, but instead of hunting big game or only gathering berries and roots, today’s professionals are often always-on, data hunter-gatherers, who carry at least one smart device—always.
“If we don’t have connectivity, it’s really a problem,” said Inzerillo, noting that many potential attendees may reconsider going to a sporting event if it means being out of pocket for several hours—if your venue doesn’t have suitable connectivity, you will not only harm customer satisfaction, but may lose visitors outright.
And such an outcome is counter to the businesses’ goals.
“Exhibition and venue people are in the customer business,” said David Dubois, president and CEO of the International Association of Exhibitions and Events (IAEE). “They want every customer to come into their building, have a great experience, and leave going, ‘Wow, isn’t that great!’”
Regarding venue Wi-Fi, Dubois emphasizes three critical aspects: quality of service, adequate bandwidth, and affordable price.
“The reality is technology augments and supports the industry and we need to embrace it 365, 24/7, and with all arms and legs,” Dubois said.
John Rissi, senior vice president of operations for PSAV, sees a challenge in the pace of technological innovation and device proliferation that is driving most venue capital-intensive network infrastructure upgrades—nobody wants to pay for it.
He says venues are asking, “What is the ROI? How do I justify spending all this money continually upgrading my network if I don’t have a return on my investment?”
From a supplier point of view, Wi-Fi pricing is still in the “Wild West” stage, as characterized by Michael Owens, managing partner of Eventgenuity and chairman of the Convention Industry Council’s (CIC) APEX Standards Committee.
“How do we pay for it? Will we ever settle on a standard model for costs?” Owens asks of the industry. “I can go to one venue this week and it’ll be ‘XYZ number of devices or number of people,’ but the next [venue] will be [based] on consumption.”
In 2005, the Boston Convention & Exhibition Center (BCEC) and the Hynes Convention Center were the first of such U.S. facilities to offer free Wi-Fi. Tchad Rogers, interim CIO of the Massachusetts Convention Center Authority (MCCA), which oversees both venues, notes that providing some of the best Wi-Fi connectivity in the industry is a source of pride and creates ROI in goodwill for clients.
“This is not a ‘nice to have;’ it is an absolute must-have,” Rogers said. “We think a lot of our shows appreciate that and continue to come back because of it.”
As chairman of the CIC, IAEE’s Dubois wears many hats and listens to people on all sides—venues, hotels, CVBs, DMCs, show organizers, and meeting planners—from around the world. He hears how much venues have spent in upgrading network infrastructures—sometimes to the detriment of other infrastructure maintenance—and knows that there are only two ways to reduce the operating deficit of a facility: Increase revenue and reduce expenses.
On the flipside are the show organizers and planners who are fully cognizant of the value of the business they bring to venues and the cities in which they are located, and they tend to want wireless connectivity that’s better, faster, and, oh yes, free.
As a result, Dubois said, negotiations for meetings and events at U.S. venues are centered on complementary discounts. But, much like 30 years ago, it’s becoming more of a seller’s market.
He generalizes that “probably 95 percent [of space suppliers] don’t comp meeting room or exhibit hall rental in Europe. You’re paying meeting room rental, you’re paying exhibit hall rental, and, by the way, ‘We’ll give you a much better Wi-Fi experience because we’ve got revenue that helps pay for all of that.’”
The current paradigm in the U.S. may shift back to that model. Possibly a totally new method may emerge from the hive mind based on co-operation, not competition.
Baseball, Hot Dogs, and Wi-Fi
MLBAM foresaw, planned, organized pan-industry cooperation, and has almost completed the execution of a systemic overhaul of the Wi-Fi infrastructure across 30 Major League Baseball (MLB) venues. Established in 2000, this wholly-owned subsidiary and limited partnership of club owners sought to centralize all Internet rights into a single entity.
Baseball and high tech isn’t as surprising a coupling as you might think. In fact, baseball has always had an almost prescient entwinement with technological innovations in media.
- The Thomas Edison Co. shot the first-known baseball footage in 1898.
- In the 1911 World Series, a mechanical scoreboard, known as the Playograph, displayed updated game statistics by operators listening to telegraph transmissions.
- The first live radio broadcast of a baseball game was in 1921; the game’s first TV broadcast came in 1939.
- Yankee Stadium was home to the first electric scoreboard in 1950.
- Dodger Stadium had the first live video scoreboard in 1980.
- In 2002, MLBAM streamed the very first live sporting event online—in response, some observers referred to the stream’s 30,000 viewers as an attempt to “break the Internet.”
- The first live-streamed video of a MLB game for mobile devices came in 2009.
- Last year, MLBAM was the No. 1 sports streaming service.
- Viewers of the 2014 World Series experienced the debut of yet another innovation: the MLBAM StatCast optical tracking and visualization technology, adding terabytes of player performance data to the narrative of every game.
- Called “The Biggest Media Company You’ve Never Heard Of” by Forbes.com in 2014, MLBAM provides tech services and distributes and delivers digital content for third-party clients in the sports, news, and entertainment markets, with clients such as the WWE, PGA Tour Live, WatchESPN, HBO Now, Sony PlayStation Vue, 120 Sports, and StubHub.
MLBAM visionaries Inzerillo and CEO Robert Bowman recognized the power of the Internet from the outset of the establishment of MLBAM and four years ago decided to institute the Wi-Fi Everywhere Plan, a $300 million initiative to bring Wi-Fi and distributed antenna systems (DAS) to every U.S. ballpark. Shortly after the age of the modern smartphone began, Bowman and Inzerillo realized connectivity and bandwidth would become huge factors in providing exceptional fan experiences.
They also saw the need for bandwidth power with next-generation applications that just simply hadn’t existed at the time.
“Not that we knew exactly how they were going to roll out or which ones would exist, but we knew some of them would,” Inzerillo said. “And that something was going to require bandwidth. We figured, strategically, it’s not wise for us to sort of hope that people figure out [Wi-Fi] or hope that teams, venues, and carriers figure it out, because without some real catalyst to drive it forward, it was just not going to happen organically. It was important to solve this for cool experiences in venues.”
Today’s professionals are often always-on, data hunter-gatherers, who carry at least one smart device—always.
What they are doing is phenomenal and a possible template for other segments of the venue industry.
MLB was the first sports league to establish a set of voluntary connectivity standards, and Bowman and Inzerillo, with a lot of cajoling and perseverance, brought everyone to the table—club owners, carriers, venues, venue managers, and IT experts—for peer-reviewed, multi-faceted, pan-industry input and co-operative execution of the Wi-Fi Everywhere Plan. Not only did everyone have some skin in the game, but these stakeholders realized that, ultimately, the fans were customers of all parties involved.
Inzerillo describes the process as “in the home stretch and there’ll still be a few ballparks that get turned up fully as 16 are in some part of the process right now. So it’s not quite done and beyond that, there’s always tweaking. This is endurance, not a sprint.”
Industry curiosity and feedback for this process has been very positive, he adds. The National Football League followed MLB nearly two years later with a similar set of standards for its stadiums.
And the payoff of increasing connectivity for the fans? Getting people more excited about the sport, making it easier to sell tickets for future games, and streaming products are some of the direct benefits Inzerillo identifies, but he is also looking on the horizon: This is not your grandpa’s game; growing a new generation of fans means finding new ways to engage them.
“We’ve all had our mediums, but this new medium happens to be very mobile and ‘very internet’ in that little ‘i’ sense where it’s a connected thing that’s not related to this very specific channel and this very specific proximity,” Inzerillo said. “You have a general purpose device, and you expect to access the world from that device.”
Enter the Spectrum and Saturation
If that device happens to be your smartphone, it operates at the 2.4 GHz and/or 5 GHz frequency of the radio spectrum. This is the radio frequency (RF) portion of the electromagnetic spectrum, which is divvyed up between personal use, private enterprise, commercial business, and state and local governments—administered by the U.S. Federal Communications Commission (FCC)—and federal entities (e.g., the branches of the military, the Department of Homeland Security, Federal Bureau of Investigation, Federal Aviation Administration), administered by the National Telecommunications & Information Administration.
Most parts of this spectrum are licensed (such as the range that cellular carriers—many of whom partner with venues—operate on) and others are unlicensed. It is this unlicensed spectrum that is of particular interest to the hospitality industry as a whole.
“Wi-Fi meeting space operates on a part of the radio spectrum that we all share, so basically it’s unlicensed,” John Rissi said. “So as Wi-Fi becomes more critical to the success of the events, we have to come up with some sort of collaborative approach [for] how to manage the local Wi-Fi networks in the same space for high-density congestion problems.”
Some in the industry have noted saturation at the 2.4 GHz range for a while. Inzerillo said there are two truisms of relevance here: There’s not enough bandwidth, and there will never be enough bandwidth.
“I don’t think that if you carved up 100 percent of the spectrum, you’d ever get an acceptable amount of bandwidth that would actually give you what you needed from a capacity standpoint,” Inzerillo said. “I think the appetite is just too large. The 2.4 GHz is definitely saturated. Every piece of white space that comes on gets saturated.”
And all those Bluetooth devices being used in venues are transmitting and communicating data via RF as well, on a frequency of 2.45 gigahertz (between 2.402 GHz and 2.480 GHz, to be exact), adding to Wi-Fi congestion.
East Coast
“The saturation of the 2.4 GHz range on Wi-Fi, we’ve known about that for years, it’s been an industry problem for quite a while,” said MCCA’s Rogers. “We’re just now starting to see problems with saturation—cross-talk—on the Bluetooth wavelength as people bring in stuff like smart watches and more smartphones, Bluetooth headsets, Bluetooth headphones, and all of those different devices. So with the Internet of Things, the impact is going to be the increase in the number of devices, leading to a corresponding increase in the noise, the cross-talk on the two different spectrums. And that is always going to be a challenge in high-density environments. When you put 25,000 people in one building, each with many devices all using the same bandwidth, the issue is not the backbone, the issue is the wireless spectrum that they’re all trying to chatter on at the same time.”
West Coast
Jeff Hardy, network engineer at The Moscone Center in San Francisco, also sees the beginnings of the impact of spectrum crunch and has concerns.
“With high-density wireless deployments, especially in keynote rooms, we could really use some more spectrum,” Hardy said. “We are able to provide high-density wireless; however, we are seeing a huge increase in connected devices and throughput. I expect this increase to continue and we have to grow with it. There comes a point at which the medium is just full. We need more spectrum to satisfy this growing need. Wi-Fi is no longer a convenience for most people, and the demand for high-speed wireless is here.”
The Third Coast
Population density and location may be the deciding factors on whether spectrum crunch is affecting venues. In the Dallas-Fort Worth area, John Winborn, CIO of the Dallas Cowboys—the home field of which is AT&T Stadium in Arlington, Texas—hasn’t seen the effects of spectrum crunch and it isn’t on his list of issues to worry about just yet, but he sees the potential.
“We’re definitely getting to the point of where we can’t add any more access points, unless they open up the spectrum a little bit. But for right now, we seem to be ahead of demand, so I think we’re in pretty good shape now,” Winborn said. “But I think [the FCC is] definitely going to have to open up more of the spectrum down the road because everything, the amount of people getting on our network is growing, and [for] the increased demand we’re going to need more spectrum… [When] I have people not being able to connect to my stadium, then it’s a problem.”
Open, unlicensed wireless spectrum availability is critical to innovation and the economic infrastructure of the U.S.
For international venue Wi-Fi consultant and former Cisco engineer Neil Reid, the spectrum crunch is a very interesting, potentially polarizing issue.
“I don’t know that I’d adopt the position that there is a spectrum crunch in Wi-Fi,” Reid said. “Mobile cellular for example does so much more with so much less. I doubt the Institute of Electrical and Electronics Engineers (IEEE) would ever believe we have enough spectrum. If we suddenly had 10 times the current allocation, would it slow efforts to be more elegant in our engineering? I would offer that something like half of all the problems I have the privilege of mitigating in the large public venue world of Wi-Fi is from deficient engineering, not from a product side but from a deployment consideration. I’d like to see vastly better engineering sequencing and methodologies on the deployment side before we make compromises in the spectrum world.”
Opening the Spectrum
But alarm bells have rung and compromises have begun. In a 2009 address to the CTIA-The Wireless Association, then-FCC Chairman Julius Genachowski warned of the looming spectrum crisis in the U.S. because of huge expected increases in data-intensive applications for unlicensed wireless spectrum use by mobile devices and licensed spectrum scarcity.
Radio frequencies, a part of the electromagnetic spectrum, are utilized for different forms of communication by propagation of radio waves. In the U.S., the use of the radio waves is regulated by the FCC to prevent interference and to allow the coordination of others who are using them for the same purposes. The two greatest problems in venue wireless networking are bandwidth network congestion and antenna interference.
In 2010, the FCC unveiled its “Connecting America: The National Broadband Plan,” which seeks to improve Internet connectivity, an increase of 500 MHz of spectrum by 2020 (as of July this year, nearly half has been opened), reallocation of spectrum, and energy management.
In March 2014, the FCC adopted new rules to increase the availability of the 5 GHz wireless spectrum and alleviate Wi-Fi congestion. Notable, from the FCC’s statement:
“The new rules will make 100 MHz of spectrum more accessible for use in homes and congested spaces like convention centers, parks, and airports and increase the potential for more unlicensed spectrum innovation…
“[This modified] the rules governing the operation of Unlicensed National Information Infrastructure (U-NII) devices operating in the 5 GHz band… U-NII devices play an important role in meeting public demand for wireless broadband service. Currently U-NII devices … are used for Wi-Fi and other high-speed wireless connections. These devices support a variety of applications including Wi-Fi hot spots and … networks to connect smartphones, tablets and laptops to the Internet…
“The rules … allow U-NII devices to better integrate with other unlicensed portions of the 5 GHz band to offer faster speeds and reduce congestion at crowded Wi-Fi hot spots such as airports and convention centers.”
Wi-Fi refers to the IEEE 802.11’s set of standards introduced in 2001. It utilizes the frequencies available on the unlicensed spectrum.
Open, unlicensed wireless spectrum availability is critical to innovation and the economic infrastructure of the U.S. The Internet of Things’ predicted explosion will depend upon this section of the RF spectrum. And the ability of venues to continue to provide the excellent customer service and fulfillment of attendee expectations will also heavily depend upon the unlicensed spectrum.
As Genachowski noted early on, “This new unlicensed spectrum will be a powerful platform for innovation… When we unleash American ingenuity, great things happen.” FM
PART III of the Venue Wi-Fi series will explore some of the issues of disruption in venue Wi-Fi networks, the impact of the recent Marriott and Smart City settlements with the FCC, recommendations that venues and suppliers would like show organizers and event planners to understand, and a short glossary that will be essential for the “Bandwidth Conversation.”
Recent Venue Upgrades
Earlier this year, the Anaheim Convention Center (ACC) in California announced a $2.5 million investment by Smart City for a Cisco-based, high-density Wi-Fi network. With an upgrade including nearly 700 access points, location-based services potential, and 10 gigabit Internet connectivity (the first convention center on the U. S. West Coast to possess such a powerful backbone) the ACC will be one of the country’s most high-tech convention centers.
John Winborn, CIO for the Dallas Cowboys, projects that by the time this year’s National Football League season starts AT&T Stadium in Arlington, Texas, will have more than 1,800 access points on its Wi-Fi network. The stadium already boasts the equivalent of 11 cell phone towers on its distributed antenna systems (DAS) network, which sees the same amount of traffic on a typical game day as that of Dallas suburb McKinney—repeatedly named one of the fastest growing cities in the country during the past 15 years.
Veteran mobility and network engineer Neil Reid, principal of Neil Reid & Associates, designed and deployed the Wi-Fi network at the brand new Avaya Stadium, home to Major League Soccer’s San Jose Earthquakes, and calls it “one of the most beautiful stadiums in the world.” Winner of the Product Innovation Award at The Stadium Business Awards this year in Barcelona for being the first cloud-enabled stadium (eliminating the need for a stadium app), this $100 million, 18,000-seat venue ups the game for immersive fan engagement with live social feed updates. Reid stresses that the technology for fan connectivity was only part of the picture of designing the facility network: critical to the entire configuration was the “back of the house operations” of owners, managers, coaches, players, and vendors Internet connectivity.
Considered by some to be the most high-tech stadium in the world, Levi’s Stadium in Santa Clara, California, was recognized at the 2015 Stadium Business Awards as Venue of the Year and awarded the Venue Technology Award for its app. This $1.2 billion home of the National Football League’s San Francisco 49ers offers free, simultaneous Wi-Fi connectivity for more than 70,000 fans, while the stadium’s app allows mobile ticketing and parking passes, in-seat ordering of food and beverages, instant replays, turn-by-turn navigation, and, most importantly, highlights which bathrooms have the shortest line.
The Massachusetts Convention Center Authority is rolling out a $2.6 million wireless infrastructure upgrade this July at the Boston Convention & Exhibition Center (Neil Reid & Associates also consulted on this project). The number of access points will increase to at least 513, transitioning the venue into a fully redundant, agile, robust high-density system. Because of the dark fiber optic ring network expansion that was recently completed, the convention and exhibition center has the ability to increase bandwidth with a phone call and cross-connect vendors with a dedicated link to their own server rooms. More than 200 iBeacons have been installed at the facility, and 100 more are in the process of being added for the venue’s geolocation system.
