On 1 April, the Italian government banned ChatGPT, an artificial intelligence system that generates human-like text and computer code. ChatGPT is trained on massive amounts of data scraped off the internet and the Italian authorities were concerned that this constituted a breach of privacy for those who owned the data. The ban continues as it investigates further.
But this ban has had an unintended consequence. One of the big questions about ChatGPT and other so-called large language models, is how humans are beginning to use them to improve their productivity (or to replace other humans entirely). The potential seems clear but nobody knows how ChatGPT is already filtering into the workplace.
Enter David Kreitmeir and Paul Raschky at Monash University in Australia, who realized that the Italian ban inadvertently created the perfect conditions to find out by looking at work patterns before and afterwards. It turns out, they say, that ChatGPT is not just being used extensively in at least one profession, but that productivity in this industry has come to depend on it.
A Question of Coding
The area of work in question is computer coding. Anecdotal evidence abounds that coders extensively use ChatGPT to develop, write and correct code but evidence beyond this is hard to get.
But Kreitmeir and Raschky realized that if programmers in Italy had begun to rely on ChatGPT, the signs should quickly become apparent when access was switched off. It turns out that the working patterns of coders is recorded in various ways by GitHub, a widely used online software repository that also manages version control.
So they downloaded data from GitHub showing the hourly coding output of 8000 programmers in Italy and in two other European countries from before and after the ban.
The results are something of a surprise. “We find that the ban of ChatGPT decreased output of Italian GitHub users by around 50% in the first two business days after the initiation of the ban,” say Kreitmeir and Raschky. “Output levels returned to normal levels after that.”
The researchers say the return to normal was probably the result of coders finding ways round the ban, which is easy to circumvent using a virtual private network or Tor Relay to access ChatGPT in other countries.
Decline and Fall
Indeed, the researchers noted an immediate increase in Google searches for “virtual private networks” the day after the ban.
“Our findings provide support for the notion that the sizable but short-lived decline in output after the ban is the result of Italian users looking for and successfully finding ways to circumvent the blocked access,” they say.
That’s interesting work suggesting that ChatGPT is much more widely used than anticipated. “We interpret these findings as first evidence that ChatGPT is already actively used for high-skilled tasks in the economy,” conclude Kreitmeir and Raschky.
Given that ChatGPT was launched only in November last year, this may just be the first trickle in a tsunami of evidence to come.
Ref: The Unintended Consequences of Censoring Digital Technology – Evidence from Italy’s ChatGPT Ban : arxiv.org/abs/2304.09339