We can categorize the environment into different types:
Deterministic and Stochastic environments: In a deterministic environment, we are certain that when an agent performs action a in state s, then it always reaches state 𝑠′. For example, let's consider a grid world environment. Say the agent is in state A, and when it moves down from state A, it always reaches state D. Hence the environment is called a deterministic environment. On other hand, in a stochastic environment instead we don't have certainly but a probability distribution over the agent's actions. Taking the grid world environment example, let's say our agent is in state A; now if it moves down from state A, then the agent doesn't always reach state D. Instead, it reaches state D 70% of the time and state B 30% of the time.
Discrete and Continuous environments: A discrete environment is one where the environment's action space is discrete. For instance, in the grid world environment, we have a discrete action space, which consists of the actions up, down, left, right. A continuous environment is one where the environment's action space is continuous. For instance, suppose we are training an agent to drive a car, then our action space will be continuous, with several continuous actions such as changing the car's speed, the number of degrees the agent needs to rotate the wheel, and so on.
Episodic and Non-episodic environments: In an episodic environment, an agent's current action will not affect future actions and in a non-episodic (also called sequential) environment, an agent's current action will affect future actions. For example, a chessboard is a sequential environment since the agent's current action will affect future actions in a chess match.
Finally, we have Single and Multi-agent environments: When our environment consists of only a single-agent, then it is called a single-agent environment, and when we have multiple agents, then it is called a multi-agent environment.
