NYC’s AI Chatbot Has Been Telling Businesses to Break the Law

NYC’s AI Chatbot Has Been , Telling Businesses to Break the Law.

New York City's AI chatbot was created to help small business owners, but it's been dispensing concerning advice, AP reports.

.

For example, many small business owners have been given inaccurate information about local policies or been encouraged to break the law.

However, the city is not removing the chatbot from its official website.

Instead, it has provided a disclaimer stating that the chatbot may "occasionally produce incorrect, harmful or biased" information.

Critics say that the situation highlights the dangers of AI being used by governments without proper guardrails.

They’re rolling out software that is unproven without oversight, Julia Stoyanovich, computer science professor and director of the Center for Responsible AI at New York University, via statement.

It’s clear they have no intention of doing what’s responsible, Julia Stoyanovich, computer science professor and director of the Center for Responsible AI at New York University, via statement.

There’s a different level of trust that’s given to government.

Public officials need to consider what kind of damage they can do if someone was to follow this advice and get themselves in trouble, Jevin West, a professor at the University of Washington and co-founder of the Center for an Informed Public, via statement.

Microsoft, which powers the chatbot, said it is working with the city "to improve the service and ensure the outputs are accurate and grounded on the city’s official documentation.".

On April 2, Mayor Eric Adams said that letting users find issues with the chatbot is just part of sorting out the new technology.

Anyone that knows technology knows this is how it’s done.

Only those who are fearful sit down and say, ‘Oh, it is not working the way we want, now we have to run away from it all together.’ I don’t live that way, Mayor Eric Adams, via statement.

Julia Stoyanovich, a computer science professor and director of the Center for Responsible AI at New York University, referred to Adams' approach as "reckless and irresponsible," AP reports.