InfoQ Dev Summit Boston: How to be a responsible developer in the age of the artificial intelligence craze

At the InfoQ Dev Summit in Boston, Akamai's Justin Sheehy gave a heartfelt keynote address on how to be a responsible developer in an era of hype around artificial intelligence (AI). The talk was aimed at software professionals who may be feeling overwhelmed by the rapid development and inflated expectations associated with AI.

Sheehy began his presentation by reminding developers that in this era, their choices matter a great deal. He highlighted three scenarios in which developers must exercise caution: in learning about the AI system, in using it wisely and safely, and in building it with minimal damage.

There are two main takeaways from my presentation. The first is a call to use one's abilities as a developer to thoughtfully evaluate claims about the systems they see in order to decide, based on evidence (rather than hype), how best to use them. The second is a call to take responsibility for your place in those systems by being personally useful, honest, and harmless to people through your work. - Justin Sheehy

The speaker emphasized that developers have power; they make decisions that can shape the trajectory of technology. However, with great power comes great responsibility - a point echoed by technology industry analyst Steve O'Grady more than a decade ago.

Sheehy then spoke about what he called the "AI era", emphasizing that the impressive advances in AI have been overshadowed by the even bigger hype surrounding it. He urged listeners not to fall for grandiose claims about modern language models, such as Google's GPT-4o or Gemini 1.5, without substantial evidence to back them up. He also noted that LLM creators are very quick to point out their shortcomings with respect to their own legal responsibilities.

He also debunked several misconceptions about these models, such as their ability to develop human-like intelligence or pass Turing tests based solely on convincing imitation of human text. According to Sheehy, they are probabilistic repetitive machines similar to parrots imitating sounds without any understanding or meaning behind those sounds.

The speaker cautioned against "bias laundering," a concept popularized by Margaret Mitchell where people tend to treat the answers generated by algorithms as objective facts without considering the potential bias inherent in the training stages. This can get companies into legal trouble if they unknowingly violate safety regulations or rights because of biased results from their systems.

In addition, Sheehy cautioned against "AI washing," where companies claim their products are equipped with advanced AI technologies, mainly for marketing purposes rather than for real improvements in functionality. This practice can divert resources from other important work and lead to dangerous erroneous decisions based on overconfidence in AI capabilities.

The speaker concluded his presentation by presenting a scheme of "coherent AI", which he believes can be applied to the developers themselves. The three "H's" of this concept are utility, integrity, and harmlessness. By adhering to these principles, developers can ensure that they create technologies that truly benefit society while minimizing potential harm.

Asked about a report at Neurips that suggested that emergent abilities in LLMs may be a mirage, Sheehy said:

It has some great conclusions, such as "putative emergent abilities disappear when other metrics are used or when statistics are improved" and "emergent abilities can be a creation of researcher choice", and this paper carefully shows that these emergent properties emerge more gradually than people might assume, depending on how they are measured... I would also point out that there is another class of supposed emergent abilities (often those that point more sharply to AGI) that are purely imaginary - they don't exist at all - but are often discussed in popular culture. I would advise people to beware of confusing these two categories.

Developers interested in learning more about the InfoQ Dev Summit series can visit the website and keep an eye out for InfoQ to release speaker recordings in the coming months.