Course: AGV Simulation with PyBullet
Session: 2
Date: August 23, 2026
Level: Beginner → Intermediate
Focus: Understanding PyBullet as the simulation engine behind our AGV project
1. What Is PyBullet?
Before we start building our AGV, we need to understand the software that will act as our virtual physics laboratory.
PyBullet is a Python module built on top of the Bullet Physics SDK. It allows us to create and control physics-based simulations of robots, objects, environments, and sensors directly from Python.
In simple terms:
PyBullet allows us to build a virtual world where a robot can move, collide with objects, sense its environment, and respond to physical forces — all through Python code.
The official PyBullet documentation describes it as a Python module for physics simulation, robotics, and machine learning. It supports articulated robot models, dynamics simulation, kinematics, collision detection, and ray-intersection queries. (GitHub)
For our course, this is extremely important because we don’t want to simply make a robot look like it is moving.
We want the robot to behave like a physical machine.
For example, if our AGV:
- accelerates,
- turns,
- hits an obstacle,
- loses traction,
- receives sensor measurements,
- follows a path,
- carries a load,
the simulation should represent these interactions through a physics engine rather than simply changing the robot’s coordinates.
2. Why Are We Using PyBullet for This Course?
Our final objective is not simply to learn PyBullet.
Our objective is to build an AGV simulation system that can gradually evolve toward a more advanced robotic system.
Our overall path looks approximately like this:
Python
↓
PyBullet
↓
Robot Model
↓
Physics
↓
Sensors
↓
Control
↓
Navigation
↓
Computer Vision
↓
AI
↓
AGV Digital TwinTherefore, PyBullet is one of the foundations of our project.
Later, we want our AGV to be able to:
- exist inside a simulated factory,
- move using its wheels,
- detect obstacles,
- receive sensor data,
- navigate toward a destination,
- follow paths,
- make decisions,
- interact with the environment,
- and eventually become part of a larger robotic/smart-factory simulation.
This is why we are learning PyBullet as an engineering tool rather than merely learning a Python library.
3. What Can PyBullet Do?
PyBullet provides several capabilities that are directly relevant to robotics.
Major capabilities include:
| Capability | What it means for our AGV |
|---|---|
| Physics simulation | Simulate gravity, motion and forces |
| Collision detection | Detect contact with walls and obstacles |
| Robot simulation | Load and simulate robot models |
| Joint control | Control robot joints and mechanisms |
| Sensors | Simulate cameras and other sensing methods |
| Kinematics | Calculate robot poses and joint configurations |
| Dynamics | Calculate motion under forces and torques |
| Ray casting | Detect objects along virtual rays |
| Python API | Control the entire simulation using Python |
| Visualization | See the simulation in a 3D environment |
PyBullet supports robot models such as URDF, SDF and MJCF and provides forward/inverse dynamics, forward/inverse kinematics, collision detection and ray queries. (GitHub)
4. PyBullet and Physics
This is one of the most important concepts in this course.
A normal 3D animation can make an object appear to move.
But a robotics simulator needs to answer a much more difficult question:
What should happen physically when the robot interacts with the environment?
Imagine that our AGV is moving forward and suddenly encounters a wall.
A simple animation might simply move the AGV’s position:
AGV position:
Before:
[ AGV ] → → →
After:
[ AGV ]But a physics simulation needs to determine what happens when the robot reaches the wall.
For example:
WALL
│
│
[ AGV ] → → │
│
The physics engine can determine that the bodies have collided and calculate the resulting physical interaction.
This is where Bullet Physics becomes important.
PyBullet provides the Python interface through which we communicate with the Bullet physics engine.
5. Physics vs. Animation
This distinction is essential for understanding our entire course.
Animation
In animation, we might say:
robot_x += 0.01This essentially means:
Move the object 1 cm.
There is no requirement for the object to obey physical laws.
Physics simulation
In a physics simulation, we define things such as:
- mass,
- gravity,
- velocity,
- forces,
- torques,
- friction,
- collision geometry,
- constraints,
- joints.
Then the physics engine calculates what happens.
Conceptually:
Forces
↓
Physics Engine
↓
Motion
↓
New PositionThis difference becomes extremely important when we start controlling our AGV.
6. PyBullet in Robotics
PyBullet is particularly useful for robotics because robots are not just 3D objects.
A robot consists of multiple interacting components.
For example, consider a simple mobile robot:
┌───────────────┐
│ AGV │
│ │
O────│ │────O
Wheel │ │ Wheel
└───────────────┘
We need to represent:
- the robot body,
- wheels,
- wheel rotation,
- mass,
- friction,
- collisions,
- sensors,
- movement,
- and eventually controllers.
PyBullet allows us to construct these elements within a physics-based environment.
This is why it is much more useful for our AGV project than simply creating a 3D model.
7. Simulating Robots
One of the major strengths of PyBullet is its ability to load robot descriptions.
A common format is:
URDF — Unified Robot Description Format
A URDF file describes the structure of a robot, including things such as:
- links,
- joints,
- visual geometry,
- collision geometry,
- mass,
- inertial properties,
- joint relationships.
Conceptually:
Robot
│
┌───────┴───────┐
│ │
Link 1 Link 2
│ │
Joint Joint
│ │
Link 3 Link 4
When PyBullet loads a robot model, it can use this information to create a physical representation of the robot.
This becomes particularly important later when we build our own AGV model.
8. Simulating Sensors
A robot without sensors is essentially blind.
Our real AGV will eventually need information about its environment.
For example:
Obstacle
█
█
█
↗ ↑ ↖
/ │ \
/ │ \
/ Sensor \
[=========================]
AGV
In a simulation, we can create virtual sensors.
Depending on the application, we can simulate things such as:
- cameras,
- depth cameras,
- ray-based distance sensors,
- collision sensors,
- contact information,
- force/torque measurements.
PyBullet also supports camera rendering and can provide RGB images, depth information, and segmentation information. (GitHub)
This capability will become particularly important later in our course when we introduce:
Computer Vision → Perception → Navigation → AI
9. Physics and Dynamics
There are two concepts that students should distinguish carefully:
Kinematics
Kinematics is mainly concerned with:
How does a robot move geometrically?
For example:
Wheel rotation
↓
Robot displacement
↓
Robot positionDynamics
Dynamics asks:
Why does the robot move this way?
For example:
Motor Torque
↓
Wheel Force
↓
Acceleration
↓
Velocity
↓
PositionDynamics therefore considers physical quantities such as:
- mass,
- force,
- torque,
- acceleration,
- friction,
- gravity,
- inertia.
PyBullet provides both forward and inverse dynamics functionality, which makes it useful for studying robotic motion beyond simple geometric transformations. (GitHub)
10. PyBullet and Python
One of the biggest advantages of PyBullet for our course is that we can control the simulator directly from Python.
Conceptually:
Python
│
│ PyBullet API
↓
PyBullet
│
↓
Bullet Physics Engine
│
↓
Simulated Robot
For example, we can eventually write Python code such as:
import pybullet as p
physicsClient = p.connect(p.GUI)Here Python is communicating with the PyBullet API.
Later, we will use Python to:
- create the simulation,
- load the environment,
- load the AGV,
- control the wheels,
- read sensor data,
- calculate navigation commands,
- process camera images,
- implement algorithms,
- and connect different modules of our robotics system.
This is one of the reasons PyBullet fits our course so well.
11. PyBullet Is More Than a 3D Viewer
This is a very important point.
A beginner might see the PyBullet GUI and think:
“PyBullet is a program that shows a robot in 3D.”
That is incorrect.
The 3D window is only the visual representation of the simulation.
The real system is closer to:
PYTHON
│
▼
PyBullet API
│
▼
Physics Simulation
│
┌────────────┼────────────┐
▼ ▼ ▼
Robot Objects Sensors
│ │ │
└────────────┼────────────┘
▼
Simulation
│
▼
Sensor Data
│
▼
Controller
│
▼
Robot Motion
This architecture is much closer to how we should think about our AGV project.
12. An Example: Our Future AGV
Imagine that our AGV is inside a warehouse.
┌─────────────────────────────────────────┐
│ │
│ █████ ███ │
│ █████ AGV → ███ │
│ │
│ █████ │
│ █████ │
│ │
└─────────────────────────────────────────┘The AGV has a destination.
The system might work like this:
Step 1 — Sensor
The sensor detects an obstacle.
Sensor → Obstacle detectedStep 2 — Perception
The software processes the sensor information.
Sensor Data
↓
PerceptionStep 3 — Decision
The navigation system determines what to do.
Obstacle
↓
Navigation Algorithm
↓
New DirectionStep 4 — Control
The controller generates commands for the wheels.
Navigation
↓
Controller
↓
Left Wheel / Right WheelStep 5 — Physics
PyBullet calculates the resulting motion.
Wheel Commands
↓
Physics Engine
↓
Robot MotionAnd then the cycle repeats.
This creates a closed-loop robotic system:
┌──────────────────────┐
│ │
▼ │
Sensor │
│ │
▼ │
Perception │
│ │
▼ │
Navigation │
│ │
▼ │
Control │
│ │
▼ │
AGV │
│ │
▼ │
Physics ─────────────────┘
This architecture will become one of the central concepts of our course.
13. Why Simulation Before the Real Robot?
Building and testing a real robot immediately can be expensive and risky.
Imagine testing a new navigation algorithm on a physical AGV.
A software bug could cause the robot to:
- hit a wall,
- damage its wheels,
- damage its payload,
- fall,
- or damage other equipment.
In simulation, we can test the same algorithm in a controlled environment.
For example:
Real Robot
Cost → High
Risk → High
Iteration → Slow
Hardware → Requiredversus:
Simulation
Cost → Low
Risk → Very low
Iteration → Fast
Hardware → Not initially requiredThis does not mean simulation completely replaces physical testing.
Instead, our goal is to use simulation to reduce the amount of expensive physical experimentation.
This leads to an important robotics concept that we will encounter later:
Sim-to-Real
The idea is to develop and test algorithms in simulation and eventually transfer them to a real robotic system.
PyBullet’s own documentation explicitly highlights robotics and sim-to-real transfer as an important use case. (GitHub)
14. Limitations of PyBullet
PyBullet is powerful, but it is not a perfect representation of reality.
This is an important engineering principle:
A simulation is a model of reality, not reality itself.
For example, a real AGV may experience:
- complex tire deformation,
- irregular floor surfaces,
- vibrations,
- sensor noise,
- imperfect motors,
- gearbox backlash,
- electrical delays,
- battery voltage variations,
- manufacturing tolerances.
A simplified simulation may not reproduce all of these effects accurately.
Therefore:
Simulation ≠ RealityInstead:
Simulation
↓
Approximation of Reality
↓
Algorithm Development
↓
Testing
↓
Real Robot
↓
ValidationPyBullet is therefore excellent for learning, prototyping, algorithm development, robotics research, and many simulation tasks, but the accuracy of the final result depends on how well the simulated model represents the real system.
15. What PyBullet Will Do in Our AGV Course
We are not learning PyBullet just to learn another Python package.
In this course, PyBullet will gradually become the simulation engine of our AGV system.
We will use it for:
Phase 1 — Simulation Foundation
PyBullet
↓
World
↓
Gravity
↓
Objects
↓
Simulation LoopPhase 2 — AGV
AGV Model
↓
Wheels
↓
Motors
↓
MovementPhase 3 — Sensors
AGV
├── Camera
├── Distance Sensor
└── Other virtual sensorsPhase 4 — Control
Sensor Data
↓
Controller
↓
Wheel Commands
↓
AGV MotionPhase 5 — Navigation
Map
↓
Path Planning
↓
Trajectory
↓
AGVPhase 6 — Vision & AI
Camera
↓
Image
↓
Computer Vision
↓
AI
↓
Decision
↓
AGVThis is where our project starts moving from a simple simulation toward a more complete robotic system.
16. The Most Important Mental Model
At this stage, I want you to remember one simple concept.
Don’t think of PyBullet as:
“A program that shows a robot.”
Think of PyBullet as:
“A programmable virtual laboratory where we can build, control, observe, and test robotic systems.”
That mental model will make the upcoming lessons much easier to understand.
17. PyBullet in One Diagram
Our complete concept can be summarized as:
PYTHON
│
▼
┌─────────────┐
│ PyBullet │
│ API │
└──────┬──────┘
│
▼
┌─────────────────┐
│ Bullet Physics │
│ Engine │
└────────┬────────┘
│
┌────────────┼────────────┐
│ │ │
▼ ▼ ▼
Robot Objects Sensors
│ │ │
└────────────┼────────────┘
│
▼
Simulation
│
▼
Sensor Data
│
▼
Control / AI
│
▼
Robot Motion
│
└──────────────►
18. What You Should Know After This Session
After completing this session, you should be able to answer the following questions:
1. What is PyBullet?
A Python interface for physics-based simulation, particularly useful for robotics, simulation, and machine learning.
2. What is Bullet?
The underlying physics engine/SDK on which PyBullet is based.
3. Why are we using PyBullet?
Because it allows us to develop and test our robotic system in a programmable physics-based environment.
4. Can PyBullet simulate robots?
Yes. It can load and simulate articulated robot models and interact with them through Python. (GitHub)
5. Can PyBullet simulate sensors?
Yes. It can provide simulated camera data and supports techniques such as ray-intersection queries; camera outputs can include RGB, depth, and segmentation information. (GitHub)
6. Does PyBullet replace a real robot?
No.
It is a simulation environment used for development, experimentation, and testing before — and alongside — physical deployment.
7. What is our ultimate goal?
Not simply:
“Learn PyBullet.”
Our goal is:
Build a functional AGV simulation and progressively integrate physics, sensors, control, navigation, computer vision, and AI into a coherent robotic system.
Key Takeaways
PyBullet = Python + Physics Simulation + Robotics
Remember these five concepts:
1. PyBullet is a robotics simulation tool.
2. Bullet provides the underlying physics engine.
3. Python controls the simulation.
4. Sensors allow the simulated robot to perceive its environment.
5. Our AGV will gradually evolve from a simple simulated robot
into a complete robotic system.Next Step → Session 3: Understanding the PyBullet Environment and Connecting Python to the Simulator
In the next session, we will move from “What is PyBullet?” to “How do we actually start PyBullet from Python?” and build the first connection between our Python program and the simulation environment.
Suggested image credits / source note for the website
The visual examples above are representative PyBullet simulation images. For technical reference, the official PyBullet Quickstart Guide is the strongest source for the capabilities discussed in this lesson. (GitHub)
