Robotic Control Interface & Manipulation Planning Library

This is a python library to operate a real or simulated robot, work with robot/world configurations, compute differentiable features, formulate and solve constrained optimization problems (for inverse kinematics, path optimization, and manipulation planning), and interfacing to various physical simulation engines. These python bindings were developed for easier access to the underlying C++ code base, esp. for teaching and students. This code base is how we, in the Learning & Intelligent Systems Lab, operate our robots.

Contents:

• Getting Started
  • Quick Install
  • Compiling from Source & Docstrings
  • Downloading the tutorial notebooks
• Tutorials
  • Basics on Configurations, BotOp & KOMO
    • Configurations
    • BotOp (Robot Operation) interface
    • KOMO: Motion Optimization
    • Starting with a real robot
    • Differentiable Features & Collision Evaluation
  • In Depth
    • Importing, Editing & Manipulating Configurations
    • KOMO: Reporting & Explaining Convergence
    • Direct Simulation Interface
    • CameraView, Offscreen (headless), and Realistic Rendering
    • Path finding example
    • Optimization (NLP formulation and solving)
• Lecture Script
  • Introduction
    • Reference material
    • Coding Getting Started
  • Scene & Robot Description
    • Transformations
    • Coordinates and Composition of Transformations
    • Scene Tree or Forest
  • Kinematics
    • Robots as Parameterized Trees
    • Forward Kinematics
    • Jacobians
    • General Concept of Differentiable Features
    • Summary: Implementing a Kinematic Engine
    • Inverse Kinematics
  • 3D Transformations, Rotations, Quaternions
    • Rotations
    • Transformations
  • Splines
  • Code References
    • NLP interface
    • Yaml-Graph Files
  • Cameras
    • Image, Camera, & World Coordinates
    • Homogeneous coordinates & Camera Matrix \(P\)
    • Calibration as Estimating \(P,K,R,t\) from Depth Data
• Old Material
  • Lab Course in Simulation
    • SimLab Exercise 1 - Basic Setup
    • SimLab Exercise 2 - Motion
    • SimLab Exercise 3 - OpenCV
    • SimLab Exercise 4 - Grasp
    • SimLab - Project Proposal and Final Presentation
  • Learning KOMO (K-Order Markov Optimization)
    • No dynamics? No velocities? How do you solve for dynamic problems?
    • What is KOMO?
    • What is KOMO – formally?
    • What are the main steps to define a KOMO problem?
    • Roughly, what does KOMO do with these specs?
    • Can KOMO only optimize paths?
    • Can I load an URDF?
    • First practical steps
    • Features are at the core
    • Varying decision variables per time slice: kinematic switches & force contacts
    • Summary: there are only a few fundamental methods in KOMO, the rest are helpers
    • Steps, phase, real time, and time optimization
    • LINKS to more documentation
  • Configuration
    • C’tor
    • Adding frames to the configuration
    • Copy, clear
    • Joint state: get/set, and selecting active joints
    • Frame state
    • Features: Anything you might want to compute from Configurations
    • Structural edits
    • Collision computations
    • Functions to create other classes (KOMO, Simulation, viewer, etc)
  • .g-files to describe robot configurations
    • Editing using kinEdit
    • Import from URDF
    • Notation to specify transformations
    • Joint types
  • Graph and .g-files
  • Simulation
    • C’tor
    • Stepping physics
    • Opening & closing the gripper, and gripping state
    • Getting simulated RGB and depth images
    • Simulation states: restarting the simulation in previous states
    • Helper: depthData2pointCloud
  • Array
• rai python API
  • ArgWord
    • ArgWord.name
  • BotOp
    • BotOp.getCameraFxycxy()
    • BotOp.getImageAndDepth()
    • BotOp.getImageDepthPcl()
    • BotOp.getKeyPressed()
    • BotOp.getTimeToEnd()
    • BotOp.get_q()
    • BotOp.get_qDot()
    • BotOp.get_qHome()
    • BotOp.get_t()
    • BotOp.get_tauExternal()
    • BotOp.gripperClose()
    • BotOp.gripperCloseGrasp()
    • BotOp.gripperDone()
    • BotOp.gripperMove()
    • BotOp.gripperPos()
    • BotOp.hold()
    • BotOp.home()
    • BotOp.move()
    • BotOp.moveAutoTimed()
    • BotOp.moveTo()
    • BotOp.setCompliance()
    • BotOp.setControllerWriteData()
    • BotOp.stop()
    • BotOp.sync()
    • BotOp.wait()
  • CameraView
    • CameraView.computeImageAndDepth()
    • CameraView.computeSegmentationID()
    • CameraView.computeSegmentationImage()
    • CameraView.getFxyCxy()
    • CameraView.setCamera()
  • Config
    • Config.addConfigurationCopy()
    • Config.addFile()
    • Config.addFrame()
    • Config.animate()
    • Config.attach()
    • Config.clear()
    • Config.computeCollisions()
    • Config.copy()
    • Config.delFrame()
    • Config.equationOfMotion()
    • Config.eval()
    • Config.feature()
    • Config.frame()
    • Config.frames()
    • Config.getCollisions()
    • Config.getDofIDs()
    • Config.getFrame()
    • Config.getFrameDimension()
    • Config.getFrameNames()
    • Config.getFrameState()
    • Config.getJointDimension()
    • Config.getJointNames()
    • Config.getJointState()
    • Config.getTotalPenetration()
    • Config.makeObjectsConvex()
    • Config.report()
    • Config.selectJoints()
    • Config.setFrameState()
    • Config.setJointState()
    • Config.setJointStateSlice()
    • Config.sortFrames()
    • Config.stepDynamics()
    • Config.view()
    • Config.view_close()
    • Config.view_focalLength()
    • Config.view_fxycxy()
    • Config.view_getDepth()
    • Config.view_getRgb()
    • Config.view_playVideo()
    • Config.view_pose()
    • Config.view_recopyMeshes()
    • Config.view_savePng()
    • Config.view_setCamera()
    • Config.watchFile()
    • Config.write()
    • Config.writeCollada()
    • Config.writeURDF()
  • ControlMode
    • ControlMode.name
  • FS
    • FS.name
  • Feature
    • Feature.description()
    • Feature.eval()
    • Feature.setOrder()
    • Feature.setScale()
    • Feature.setTarget()
  • Frame
    • Frame.addAttributes()
    • Frame.getAttributes()
    • Frame.getJointState()
    • Frame.getMeshPoints()
    • Frame.getMeshTriangles()
    • Frame.getPosition()
    • Frame.getQuaternion()
    • Frame.getRelativePosition()
    • Frame.getRelativeQuaternion()
    • Frame.getRotationMatrix()
    • Frame.getSize()
    • Frame.info()
    • Frame.setAttribute()
    • Frame.setColor()
    • Frame.setContact()
    • Frame.setJoint()
    • Frame.setJointState()
    • Frame.setMass()
    • Frame.setMeshAsLines()
    • Frame.setParent()
    • Frame.setPointCloud()
    • Frame.setPose()
    • Frame.setPosition()
    • Frame.setQuaternion()
    • Frame.setRelativePose()
    • Frame.setRelativePosition()
    • Frame.setRelativeQuaternion()
    • Frame.setShape()
    • Frame.unLink()
  • ImpType
    • ImpType.name
  • JT
    • JT.name
  • KOMO
    • KOMO.addControlObjective()
    • KOMO.addInteraction_elasticBounce()
    • KOMO.addModeSwitch()
    • KOMO.addObjective()
    • KOMO.addQuaternionNorms()
    • KOMO.addTimeOptimization()
    • KOMO.clearObjectives()
    • KOMO.getFeatureNames()
    • KOMO.getForceInteractions()
    • KOMO.getFrameState()
    • KOMO.getPath()
    • KOMO.getPathFrames()
    • KOMO.getPathTau()
    • KOMO.getPath_qAll()
    • KOMO.getReport()
    • KOMO.getT()
    • KOMO.initOrg()
    • KOMO.initPhaseWithDofsPath()
    • KOMO.initRandom()
    • KOMO.initWithConstant()
    • KOMO.initWithPath_qOrg()
    • KOMO.initWithWaypoints()
    • KOMO.nlp()
    • KOMO.report()
    • KOMO.reportProblem()
    • KOMO.setConfig()
    • KOMO.setTiming()
    • KOMO.updateRootObjects()
    • KOMO.view()
    • KOMO.view_close()
    • KOMO.view_play()
  • NLP
    • NLP.evaluate()
    • NLP.getBounds()
    • NLP.getDimension()
    • NLP.getFHessian()
    • NLP.getFeatureTypes()
    • NLP.getInitializationSample()
    • NLP.report()
  • NLP_Factory
    • NLP_Factory.setBounds()
    • NLP_Factory.setDimension()
    • NLP_Factory.setEvalCallback()
    • NLP_Factory.setFeatureTypes()
    • NLP_Factory.testCallingEvalCallback()
  • NLP_Solver
    • NLP_Solver.getOptions()
    • NLP_Solver.getTrace_J()
    • NLP_Solver.getTrace_costs()
    • NLP_Solver.getTrace_phi()
    • NLP_Solver.getTrace_x()
    • NLP_Solver.reportLagrangeGradients()
    • NLP_Solver.setOptions()
    • NLP_Solver.setProblem()
    • NLP_Solver.setSolver()
    • NLP_Solver.setTracing()
    • NLP_Solver.solve()
  • NLP_SolverID
    • NLP_SolverID.name
  • NLP_SolverOptions
    • NLP_SolverOptions.dict()
    • NLP_SolverOptions.set_damping()
    • NLP_SolverOptions.set_maxStep()
    • NLP_SolverOptions.set_muInc()
    • NLP_SolverOptions.set_muInit()
    • NLP_SolverOptions.set_muLBDec()
    • NLP_SolverOptions.set_muLBInit()
    • NLP_SolverOptions.set_muMax()
    • NLP_SolverOptions.set_stepDec()
    • NLP_SolverOptions.set_stepInc()
    • NLP_SolverOptions.set_stopEvals()
    • NLP_SolverOptions.set_stopFTolerance()
    • NLP_SolverOptions.set_stopGTolerance()
    • NLP_SolverOptions.set_stopTolerance()
    • NLP_SolverOptions.set_verbose()
    • NLP_SolverOptions.set_wolfe()
  • OT
    • OT.name
  • PathFinder
    • PathFinder.setExplicitCollisionPairs()
    • PathFinder.setProblem()
    • PathFinder.solve()
    • PathFinder.step()
  • ST
    • ST.name
  • SY
    • SY.name
  • Simulation
    • Simulation.addImp()
    • Simulation.addSensor()
    • Simulation.closeGripper()
    • Simulation.depthData2pointCloud()
    • Simulation.getGripperIsGrasping()
    • Simulation.getGripperWidth()
    • Simulation.getGroundTruthPosition()
    • Simulation.getGroundTruthRotationMatrix()
    • Simulation.getGroundTruthSize()
    • Simulation.getImageAndDepth()
    • Simulation.getScreenshot()
    • Simulation.getState()
    • Simulation.getTimeToMove()
    • Simulation.get_q()
    • Simulation.get_qDot()
    • Simulation.loadTeleopCallbacks()
    • Simulation.openGripper()
    • Simulation.pushConfigurationToSimulator()
    • Simulation.selectSensor()
    • Simulation.setMoveto()
    • Simulation.setState()
    • Simulation.step()
  • SimulationEngine
    • SimulationEngine.name
  • SolverReturn
    • SolverReturn.dict()
  • compiled()
  • depthImage2PointCloud()
  • getStartGoalPath()
  • params_add()
  • params_clear()
  • params_file()
  • params_print()
  • raiPath()
  • setRaiPath()

Indices and tables

• Index

• Module Index

• Search Page