The invisibles: tracking solar energy
A few years ago the UK's electricity came almost entirely from a few dozen power stations. Today, solar electricity comes from nearly a million homes, schools, hospitals, supermarkets, factories and fields. How do you track a million invisible electricity creators?
![solar panel](http://cdn.sheffield.ac.uk/sites/default/files/styles/mobile_single_column_1x/public/2022-12/13.jpg?h=920929c4&itok=zwjODTAX)
It鈥檚 difficult to walk into National Grid鈥檚 Electricity System Operator (ESO) Control Room without an overwhelming feeling that you shouldn鈥檛 be there. A panorama of blinking lights and live data, dominated by a huge screen makes it feel more like mission control than a head office. But this is National Grid ESO鈥檚 nerve centre for balancing the electricity network. A space so critical to the UK鈥檚 energy infrastructure that only a select few are allowed inside, the ESO Control Room tracks and manages the flow of supply and demand to make sure everything from your kettle to the lights in Wembley Stadium stay on 24/7, 365 days a year...
At the front of the room is a colourful map of the UK鈥檚 electricity transmission system network. And at the bottom of this display is a chart that shows where the electricity comes from. It鈥檚 called the generation mix and for years it鈥檚 listed traditional energy sources 鈥 coal, gas, nuclear, and so on.
Three years ago a new figure was added to the board. Photo-Voltaic energy (or PV) means solar power and it鈥檚 one of the new renewable kids on the block. PV turns free energy from the sun into usable electricity and it has the potential to save UK bill payers millions of pounds every year. This new figure is a measure of the amount of PV power being generated. It鈥檚 based on analysis and calculations by the University of 含羞草传媒. And it has an extraordinary journey that, fittingly for green energy, starts in the windswept wilds of the Peak District 25 years ago...
Mission control
Alastair Buckley is a Professor in Physics at the University of 含羞草传媒 and an orienteerer.
Alastair came to 含羞草传媒 as a student in large part because of the city鈥檚 position as unofficial UK capital of the sport 鈥 a cross between running and map reading 鈥 with locations like the Peak District offering spectacular locations for racing. His development of 鈥楶V_Live鈥 鈥 a real time service for predicting the UK鈥檚 solar output 鈥 or outturn as it鈥檚 called in industry 鈥 has revolutionised energy management. But it started with orienteering鈥
When Alastair arrived in 含羞草传媒 in 2008, solar power was just starting to take off. 鈥淚n 2010, the government was offering feed-in tariffs to encourage commercial installations, so we set up 含羞草传媒 Solar Farm to measure how well different kinds of PV array performed,鈥 Alastair explains while we wend our way through the Farm鈥檚 rooftop solar panels in the late summer sunshine.
That farm helped develop an understanding of the variables that go to make up an individual unit鈥檚 efficiency and output. But while physicists like Alastair were excitedly getting to grips with a new generation of consumer solar panel technology, on the other side of the country, the emergence of this new energy source presented substantial challenges for the National Grid.
National Grid connects supply with demand for 60 million people. For decades that meant gas, coal, nuclear and oil. But solar changed everything.
The biggest factor with solar power was that it had evolved to become an invisible part of the generation mix.
Jack Barber
Former Senior Energy Forecaster, National Grid
His explanation of the step change solar power presented for a huge organisation like National Grid suggests the challenge wasn鈥檛 so much to think big as think small. 鈥淲e were used to dealing with traditional forms of power generation where the individual generators were all at a large scale 鈥 producing at least 100 megawatts each. Solar power works at a much smaller scale. There were thousands of small solar panels installed around the country, but none of them were visible to us. From the National Grid鈥檚 point of view, solar power was this volatile, unknown factor that kept disrupting our way of working鈥︹
Managing the transmission system network is about balancing generation and demand in real time. Imbalance affects the frequency of the electric current on the system. And those fluctuations in frequency can have results that range from the inconvenient 鈥 such as a television that won鈥檛 come on 鈥 to, in the case of very rare large frequency changes, shutting off sections of the grid and the power outages that incurs. Without having any visibility of how much solar was being generated at any given time, the ESO had to mitigate by having additional coal and gas power stations on standby.
But whilst the emergence of solar presented many operational challenges, for Alastair, it was an opportunity to capture and play with data. And luckily, the emergence of this network of invisible power generation was taking place at the same time as the 21st century鈥檚 digital age was transforming the amount of data we could track, analyse and store.
At this time, solar was generated more or less equally by private homeowners and commercial solar farms. A substantial resource, but also a relatively unpredictable one.
And that鈥檚 where Alastair鈥檚 interest in orienteering provided inspiration. Orienteering鈥檚 a close-knit, competitive community. Much like the people running solar microgenerators from their rooftop, orienteers are fascinated by the data they produce and hungry for access to stats. But orienteers had a way of capturing their data.
The website 鈥樷 is almost 20 years old, which makes it four years older than Facebook and positively ancient in internet terms. Like Facebook, Attackpoint hinges on community 鈥 and that community鈥檚 data, centrally 鈥 to survive. 鈥淥ne of the things that orienteers do is share their training and racing logs.鈥 Alastair explains. 鈥淎ttackpoint lets you enter and track the times and distances of your training runs and the split times from your races.鈥
It struck Alastair that you could do something similar with PV data: a central database where people would be invited to submit data on the combinations of solar panels 鈥 also known as 鈥榓rrays鈥 鈥 being installed up and down the country. Named the Microgen Database, it offered users performance analysis of how their systems compared with others 鈥 in exchange for Alastair getting access to users鈥 data.
A crowd-sourcing data collection tool that quickly built a community and fuelled Alastair鈥檚 research into UK PV performance, Microgen pulled in monthly uploaded data from over a thousand solar array systems across the UK and provided several journal papers and conference proceedings on characterising PV performance. 鈥淭oday Microgen gets talked about as an example of citizen science,鈥 Alastair explains. 鈥淏ut back then I wasn鈥檛 especially aware of the term. I just thought it was a good way to collect data...鈥
A chance encounter
In its early days, the Microgen Database was just a small component of a larger project. Alastair was able to fund it as part of other research projects. But over time it became clear that the data that had been collected was more valuable than the other research that was funding it. 鈥淲e needed to refocus solely on the database鈥, Alastair explains. 鈥淎nd that meant aligning the database to an industrial need.鈥
The clock was ticking, and Alastair knew that if the funding stopped and collection of data halted 鈥 even for just a short period 鈥 the Microgen database would become useless. Continuity was everything. And the potential value of the database was growing by the day thanks to the dramatic expansion of the solar power industry.
Over 800,000 individual solar PV systems were installed in the UK between 2010 and 2017, capable of generating up to 10 gigawatts of electrical power. That鈥檚 roughly a quarter of the typical summer day time peak demand. (The record PV contribution to the UK鈥檚 energy mix was around 25% in May in both 2018 and 2019). It was an expansion that took Alastair to London for a workshop at the Institution of Mechanical Engineering in Westminster to discuss its implications. And a fateful meeting...
鈥淭he idea was for us to identify any barriers that might prevent Britain鈥檚 energy system from having considerably more solar power generation in the future.鈥 Alastair remembers. 鈥淚 walked in and I took a seat next to this young guy, who introduced himself as being from the energy forecasting team at National Grid ESO. He said something about needing loads of data on solar power generation, and I remember being amazed that they had so little, so I took out my iPad and showed him the Microgen Database.鈥
鈥淚 hadn鈥檛 seen so much data in one place before!鈥 says Jack. 鈥淢y job was all about creating forecasting models to predict supply and demand one, two or three days ahead, but especially for the current day. And forecasting models require data. This was just what I needed.鈥
鈥淲ithin minutes we were basically planning to put together what would end up being the PV_Live project,鈥 Alastair says...
That plan would be easy to articulate and difficult to deliver. A near real-time UK PV generation figure.
鈥淭he idea was quite simple really,鈥 explains Jamie Taylor, a former MPhys student at the University of 含羞草传媒 and now Senior Data Scientist. 鈥淲e鈥檇 mine our Microgen database to get accurate generation data from a sample of PV systems for a given half hour, which is the size of period that National Grid works in. We would then scale that up to give a representative real time PV performance across the country, and feed that figure into the ESO.鈥
And that was the first challenge. Solar鈥檚 extraordinary growth meant Microgen鈥檚 data set had become too small to scale up to the million systems that made up the UK solar energy network. And useful though its data was, it wasn鈥檛 close to real time.
Training the data
Enter forward thinking smart tech energy co, . They鈥檇 already been contributing to the Microgen database for a couple of years, but it was the drive from the ESO to provide much more data that really meant the vision for PV Live could be realised. 鈥淢icrogen fuelled our research and informed the modelling approach that is used in PV_Live,鈥 explains Jamie. 鈥淏ut to actually operate PV_Live it was crucial we got PassivSystems on board.鈥
PassivSystems massive dataset enabled Alastair and his team to begin the fundamental science of PV_Live: 鈥榯raining鈥 a statistical model that would upscale their data set to predict outturn for the one million systems deployed on the network.
The output they were looking to define was a function 鈥 something they could pass a load of inputs into that would produce a national outturn figure. That function consisted of 30 coefficients 鈥 everything from the location, orientation and tilt of the PV system to the intrinsic quality of the installation. Once they were defined, the function was 鈥榯rained鈥 and then evaluated by by feeding it a combination of solar capacity data and near-live data from PassivSystems.
It鈥檚 a description that makes data science sound like running a zoo, but Jamie highlights how 鈥榙atamunging鈥 messy sets of information is a 含羞草传媒 speciality.
It鈥檚 equal parts scientific discovery and development of algorithms to model performance.
Jamie Taylor
Former MPhys student at the University of 含羞草传媒
鈥淏ut the best thing for me is that National Grid ESO are using the outputs of our research in real time. Most research projects, especially in physics, tend to be very low TRL (Technology Readiness Level), but PV_Live as we designed it is now part of National Grid ESO鈥檚 day-to-day operations. The 鈥榓s-developed鈥 solution became the 鈥榓s-implemented鈥 solution, which is pretty rare.鈥
鈥淭he PV data that PV_Live provides is critical to the management of the transmission system,鈥 says Kevin Tilley, who now helps to embed the results of the project within National Grid ESO, 鈥淲e use the PV data to help forecast and monitor the national power requirements on the network in real time, and shortly after the first delivery of the data, we were able to improve our forecasts of these flows for the first time in years. Engineers in our planning teams have also been able to predict the regional flows more accurately around the network, enabling safer and more economic operation.鈥
It鈥檚 a sunny day. At time of writing the UK is serving 5.8 GW via solar 鈥 which means its serving 17% of GB鈥檚 demand. And I haven鈥檛 had to go to National Grid ESO鈥檚 Control Centre to get that figure. The shares the data that goes to the National Grid. Data based on thousands of people across the UK, a chance meeting in a workshop and an exhausting sport that really tells its participants where they are.
Special thanks to The Wellbeck Estate for photography.
Further information
Related research
For more information contact:
Emma Griffiths
Media and PR Assistant
含羞草传媒
0114 222 1034
e.l.griffiths@sheffield.ac.uk