Ethical Aspects of Using AI in Products and Organizations

Do you discuss the ethical aspects of the features and products you build? AI changes the way we work. But it does not remove our responsibility for what we create and deploy.

Artificial intelligence is increasingly helping us make decisions, analyze data, create content, design solutions, write code, and automate work. With the rapid development of generative AI, these capabilities are expanding at an impressive pace.

In organizations, we therefore ask: What can we do with AI? How can we use it faster? Where can it bring the greatest efficiency gains? What value can it create for customers and for the business?

What I observe, however, is that we ask less often: Should we do it?

And this is where the conversation about ethics begins.

My academic background also includes philosophy. I have spent most of my professional career in the world of technology, agility, and product management. Perhaps this is why ethics has never felt like an abstract branch of philosophy to me. I am primarily interested in it as a way of looking at decisions: what values lie behind them, who they affect, what consequences they may have, and whether we are prepared to take responsibility for those consequences.

Together with Patricia Kong, I explored this topic in the whitepaper Ethics in Product Management,[1] published by Scrum.org. Our starting point was a simple observation: ethics does not suddenly become relevant only when we build a product that uses artificial intelligence. Product decisions have always had an ethical dimension. Products influence people’s behavior, their choices, and their access to information or services. We decide what data we collect, what we optimize for, what behaviors we encourage, and who may benefit from or be harmed by the solutions we create.

AI did not create the problem of ethics.

What AI has changed is the speed and scale at which the consequences of our decisions can emerge. It also means that many of these questions remain just as important, if not more so.

Ethics Does Not Start with AI

When we start talking about the ethics of artificial intelligence, terms such as bias, fairness, explainability, privacy, accountability, or, more broadly, Responsible AI quickly come up.

It is also worth asking a more fundamental question: what does ethics actually mean in the context of technology and products?

In their day-to-day work, organizations look at decisions from several different perspectives.

One of them is technological feasibility: can we build it? Another is whether we are solving a real problem and creating value for customers. Will customers want to use it? Does the investment make sense for the organization?

Another perspective involves law, regulation, security, data protection, and compliance. In other words: are we allowed to do it?

In my view, we also need the question raised earlier: should we do it?

Organizations do not always ask this question and this is where we enter the field of ethics.

Let us look at it from another angle. Not every feature we are capable of building should be built. Not everything that can generate profit should necessarily be implemented. And even compliance with the law does not automatically tell us whether a particular decision is ethically sound.

The law sets boundaries. Ethics, however, helps us make decisions in the vast space that exists within and beyond those boundaries.

There are different ethical theories and frameworks, which means that the same decision can look very different depending on the perspective we adopt. We discuss these approaches in more detail, with examples, in Ethics in Product Management.

Personalization or Manipulation?

Imagine a product that, with the help of AI, can predict with a high degree of accuracy which message is most likely to persuade a particular user to make a purchase.

From a technological perspective, this may be entirely feasible.

From a business perspective, it may look like a success. Let us assume that conversion increases and revenue grows with it.

The way the system operates may also be fully compliant with applicable regulations.

And yet one question remains: at what point does personalization become manipulation?

If we know that a user is in a particular emotional state, should we use that knowledge to make a sale more likely?

If an algorithm discovers that creating anxiety increases engagement, should we optimize the product in that direction?

And if a recommendation system can keep a user in an app 20% longer, is that necessarily a good outcome?

Who Are We Creating Value For?

This leads to one more question I would add to the earlier ones: For whom?

A product may create value for the company while reducing value for the user.

A solution may be valuable for some customers while excluding a particular group.

An AI-based recruitment system may reduce the time needed to review hundreds of applications from several days to just a few minutes. But do we know on what basis it rejects candidates? Are certain groups being systematically assessed less favorably? And does a rejected candidate have any way to challenge the decision?

The question “is this ethical?” rarely has a simple yes-or-no answer. Context and perspective matter. Different ethical frameworks can help us look at the same product or decision from different points of view.

Finding one universal answer may be difficult. It is still worth asking questions that broaden the perspective:

  • Who benefits?
  • Who bears the loss?
  • Who bears the risk?
  • Who might we have overlooked?
  • Do the people affected by the decision understand its consequences?
  • Can they challenge the decision?
  • Who is accountable for its consequences?

What Does AI Change?

Many of the problems we discuss in the context of AI existed long before AI became widespread. Discrimination, manipulation, misuse of data, and lack of transparency did not emerge with large language models.

AI does, however, change several aspects of these problems, including their scale, speed, complexity, and the appearance of objectivity.

For example, an output generated by a system may look like the result of a neutral, mathematical analysis. But the model works with specific data, has been designed in a particular way, and operates within a context defined by people.

In other words, AI does not remove human choices from the process — at least not yet.

Data: If We Can Use It, Should We?

AI depends on data. Generative AI also makes it extremely easy to transfer documents, fragments of conversations, customer information, or code into external tools.

Imagine an employee who receives a lengthy document from a client. They paste it into a public generative AI tool and quickly receive an excellent summary.

They have just saved two hours of work.

Was that a good thing?

From an efficiency perspective, probably yes.

But what was contained in the document? Did the employee have the right to share that data with an external system? Did the client know that their information would be used in this way?

This is why the responsible use of AI cannot be reduced to choosing the right model.

We also need people who understand how to use AI responsibly, as well as clear organizational rules for data and tools.

Bias: AI Is Not Necessarily a Neutral Judge

Another frequently discussed issue is bias.

AI systems learn from data. If that data reflects historical inequalities, underrepresents certain groups, or is shaped by particular assumptions, the system’s outputs may reproduce those patterns.

Bias does not exist only in the data.

It can appear much earlier: when we choose the problem we want to solve, define variables, decide what success looks like, or determine where in the process a human should be replaced by automation.

This is why fairness is not only a concern for technical teams or data specialists.

It is also a product concern.

In its Recommendation on the Ethics of Artificial Intelligence[2], UNESCO highlights, among other principles, fairness and non-discrimination, privacy, transparency, accountability, and human oversight as key elements of responsible AI. It also firmly grounds these principles in the protection of human rights and human dignity.

Transparency: Should the User Know?

Imagine contacting customer support.

The conversation feels completely natural. You receive answers, follow-up questions, apologies, and even messages that sound empathetic.

Does it matter whether there is a human on the other side?

In many situations, perhaps not.

But do we have the right to know?

Since 2 August 2026, some of these issues in the European Union have no longer been purely ethical questions. The transparency requirements of the AI Act[3] require, among other things, that users be informed in certain situations when they are interacting with an AI system, and that certain AI-generated or AI-manipulated content be labelled.

This is a good example of the relationship between law and ethics. The law can establish a minimum level of transparency.

A person responsible for a product should still ask questions that go beyond legal compliance.

Does the user understand what they are interacting with?
Could the way the interaction is designed create a misleading impression?

Is the information genuinely visible, or does it only formally satisfy the requirement?

Importantly, transparency is not simply about adding the right sentence to the terms and conditions. Its purpose should be to enable people to make informed decisions.

Human Involvement Does Not Solve Everything

A common response to AI-related risk is: “A human still makes the final decision.”

But that is not always enough.

Imagine a system that analyzes one hundred cases and provides a recommendation for each one. A person reviews the results for several hours, and the system’s recommendations turn out to be correct in the vast majority of cases.

What happens with case number 83?

Will the person make an independent assessment? Perhaps yes, perhaps not.

Human involvement in the decision-making process — often referred to as human in the loop — only works when the person genuinely has the ability, competence, time, and authority to challenge the AI recommendation. Otherwise, human oversight can become little more than a formal step in the process.

Responsibility therefore cannot be reduced to simply placing a human at the end of an automated workflow.

We also need to design how humans and AI work together.

Who Is Responsible When AI Gets It Wrong?

In AI-enabled solutions, responsibility can easily become blurred. Consider a situation where the model comes from an external provider, a team integrates it into a product, the person responsible for the product decides to use it, and legal, security, and business approvals are involved along the way. As long as everything works, this distribution of responsibility may not raise major concerns. The problem starts when an AI-supported decision causes real harm.

At that point, “the system recommended it” cannot be an acceptable answer.

And this is not a purely hypothetical situation. In recent years, disputes have emerged around AI systems used in recruitment, where one of the key questions has been who should be held responsible for the consequences of technology-supported decisions: the system provider or the organization using it.[4]

That is why accountability matters so much. Who makes the final decision? Who monitors how the system operates? Who has the responsibility to act when something goes wrong?

One of the fundamental principles of responsible AI is that human responsibility for the use of the system must remain in place. UNESCO explicitly states that AI should not remove ultimate human responsibility and accountability.

What Is the Responsibility of the Person Managing the Product?

A person responsible for a product does not need to be an expert in language models, law, or ethics. Nor should they be solely responsible for every aspect of Responsible AI.

At the same time, product responsibility sits at a particularly important intersection, where user needs, business goals, technological possibilities, data, risks, legal constraints, and decisions all come together.

This is why the responsibility of the person managing the product cannot end with the question:

Can we use AI to solve this problem?

We should also ask:

  • Why do we want to use AI?
  • What should improve as a result, and for whom?
  • What new risks are we creating?
  • How will we know whether the solution is working as intended, and what will we do if it is not?

AI clearly changes the technology we work with, but it does not remove our responsibility to understand the problem, the stakeholders, the values involved, and the consequences of the decisions we make.

AI Ethics Cannot Be the Responsibility of One Person

It would be wishful thinking to assume that having a “responsible Product Manager” is enough.

If an organization rewards speed, revenue growth, and cost reduction, it may be difficult to expect one individual to stop the development of a feature because they have identified an ethical concern. Product Managers and Product Owners play important decision-making roles, but organizational structures, leadership, internal processes, and procedures strongly influence whether responsible practices are actually followed. Research conducted by UC Berkeley supports this point.[5] The study included 25 interviews and a survey of 300 people working in product management. Seventy-seven percent of respondents reported a lack of clarity around the responsible use of GenAI, while fewer than one in five had incentives linked to responsible AI use. Organizational support from leadership was associated with more frequent use of responsible practices.

If employees do not know which GenAI tools they are allowed to use, responsible handling of company or customer data is at risk.

If it is unclear who can approve the deployment of a high-risk system or where concerns should be raised, the phrase “Responsible AI” will not change much on its own.

The U.S. National Institute of Standards and Technology (NIST), in its AI Risk Management Framework,[6] treats AI risk management as an organizational and lifecycle-wide concern, rather than as a single check performed before deployment. NIST has also developed a dedicated profile for generative AI that addresses risks and management practices specific to this technology.

Organizations therefore need more than broadly stated values. They also need mechanisms that make those values actionable: clear rules for the use of data, criteria for identifying AI use cases that require additional review, awareness of model limitations, a defined process for raising concerns, and ongoing monitoring after deployment.

Above all, they need a culture in which the question “should we do this?” is treated as part of responsible decision-making.

Not every organization needs a large AI ethics department. Sometimes, a few well-framed questions are enough to start.

These questions will not resolve every dilemma. They can, however, open a meaningful conversation about the ethics of AI-based solutions and, more broadly, about the ethics of the features and products we create.

Responsible Use of AI Does Not Mean Less Innovation

Discussions about ethics are sometimes seen as an attempt to slow down technology.

Responsible use of AI does not mean giving up experimentation or avoiding risk. Nor does it mean that we need to find a perfect answer to every possible question before making a decision.[7]

Responsible AI means looking consciously at the consequences of our decisions, including those that are not captured directly by business success metrics.

AI creates enormous opportunities. It allows us to build more features, and even entire products, often much faster than was previously possible.

But as our capabilities grow, so does the importance of asking how we use them.

There are still decisions we cannot simply delegate to AI.

The point is not to turn every product decision into a philosophical debate or to create another procedure to tick off. What matters is creating space where, alongside questions such as “can we do it?”, “does it make business sense?”, and “is it legally compliant?”, we also regularly ask: “should we do it?” and “for whom?”

Perhaps this is where the conversation about AI ethics in products and organizations should begin.

P.S. This article was not generated entirely by AI. Generative AI models were used to check spelling and grammar, support editing, suggest possible revisions, and help identify additional sources and research on the topic. The main image for the article was first generated with AI using my photograph and then manually retouched. The English translation of the article from the original Polish version was produced with AI.

Facebook
Twitter
LinkedIn

More To Explore

decorative

The Overloaded Leader

There’s no shortage of training and frameworks on decision-making, prioritization, or time management. But what if the real issue lies elsewhere? What if the challenge is not a lack of knowledge, but a cognitively overloaded leader?

Read More »