{"id":258,"date":"2026-10-27T18:30:00","date_gmt":"2026-10-27T13:00:00","guid":{"rendered":"https:\/\/www.graveiensai.com\/blog\/?p=258"},"modified":"2026-10-27T16:01:47","modified_gmt":"2026-10-27T10:31:47","slug":"hand-pose-annotation","status":"publish","type":"post","link":"https:\/\/www.graveiensai.com\/blog\/hand-pose-annotation\/","title":{"rendered":"Hand Pose Annotation for Robot Learning: Keypoints, Hand-Object Labels and QA"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\"><i>Last updat\u00aded: Octo\u00adber 2026<\/i><\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Key takeaways<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Hand pose anno\u00adta\u00adtion marks the joints of the hand in each frame, so a mod\u00adel can learn how peo\u00adple shape and move their hands dur\u00ading a task.<\/li>\n\n\n\n<li>For robot learn\u00ading, key\u00adpoints alone are rarely enough: add con\u00adtact, grasp type and object state to cap\u00adture the hand-object inter\u00adac\u00adtion.<\/li>\n\n\n\n<li>Choose 2D key\u00adpoints for image-space mod\u00adels and 3D key\u00adpoints when the pol\u00adi\u00adcy needs depth; 3D usu\u00adal\u00adly needs cal\u00adi\u00adbrat\u00aded mul\u00adti-view or depth cap\u00adture.<\/li>\n\n\n\n<li>Occlu\u00adsion is the main qual\u00adi\u00adty prob\u00adlem, so write rules for hid\u00adden joints before label\u00ading starts.<\/li>\n\n\n\n<li>Good cap\u00adture makes label\u00ading cheap\u00ader: hands in frame, steady light\u00ading and syn\u00adchro\u00adnized cam\u00aderas reduce rework.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Robots that manip\u00adu\u00adlate objects need to learn from hands. Ego\u00adcen\u00adtric and mul\u00adti-view video shows those hands at work, but a pol\u00adi\u00adcy can only use what the labels describe. This guide is for robot\u00adics and com\u00adput\u00ader vision teams plan\u00adning hand pose anno\u00adta\u00adtion for manip\u00adu\u00adla\u00adtion, grasp\u00ading or imi\u00adta\u00adtion learn\u00ading. Most search results on the top\u00adic are research papers and pub\u00adlic datasets; this guide turns that work into a prac\u00adti\u00adcal spec. It cov\u00aders 2D vs 3D key\u00adpoints, hand-object inter\u00adac\u00adtion labels, label tax\u00adonomies by task, tools and for\u00admats, QA checks and cap\u00adture tips that make label\u00ading faster.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><b>Want to see the out\u00adput first?<\/b> <a href=\"https:\/\/www.graveiensai.com\/contact-us\">Request a sam\u00adple anno\u00adtat\u00aded clip set.<\/a><\/p>\n\n\n\n<h2 class=\"wp-block-heading\">What is hand pose annotation?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Hand pose anno\u00adta\u00adtion is the label\u00ading of hand joint posi\u00adtions, usu\u00adal\u00adly the wrist and fin\u00adger joints, in images or video frames. Each key\u00adpoint gets a posi\u00adtion and often a vis\u00adi\u00adbil\u00adi\u00adty flag. For robot learn\u00ading, teams usu\u00adal\u00adly add inter\u00adac\u00adtion labels such as con\u00adtact, grasp type and object state, so the data shows not just where the hand is but what it is doing.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Most projects use a fixed key\u00adpoint lay\u00adout with the wrist plus joints along each fin\u00adger, com\u00admon\u00adly 21 points per hand. Keep the same lay\u00adout, joint order and nam\u00ading across the whole dataset, because down\u00adstream code depends on them.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">2D vs 3D hand keypoints: which do you need?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Choose 2D key\u00adpoints when your mod\u00adel works in image space, such as hand detec\u00adtion or action recog\u00adni\u00adtion from a sin\u00adgle cam\u00adera. Choose 3D key\u00adpoints when the pol\u00adi\u00adcy rea\u00adsons about depth, reach or grasp geom\u00ade\u00adtry. 3D labels usu\u00adal\u00adly require cal\u00adi\u00adbrat\u00aded mul\u00adti-view or depth cap\u00adture, which rais\u00ades cap\u00adture and label\u00ading effort.<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><thead><tr><th>Option<\/th><th>Best for<\/th><th>Cap\u00adture needs<\/th><th>Effort<\/th><\/tr><\/thead><tbody><tr><td>2D key\u00adpoints<\/td><td>Detec\u00adtion, track\u00ading, action recog\u00adni\u00adtion<\/td><td>Sin\u00adgle RGB cam\u00adera<\/td><td>Low\u00ader<\/td><\/tr><tr><td>2D key\u00adpoints + vis\u00adi\u00adbil\u00adi\u00adty<\/td><td>Ego\u00adcen\u00adtric video with fre\u00adquent occlu\u00adsion<\/td><td>Sin\u00adgle RGB cam\u00adera<\/td><td>Mod\u00ader\u00adate<\/td><\/tr><tr><td>3D key\u00adpoints<\/td><td>Grasp geom\u00ade\u00adtry, retar\u00adget\u00ading to robot hands<\/td><td>Cal\u00adi\u00adbrat\u00aded mul\u00adti-view or RGB\u2011D<\/td><td>High\u00ader<\/td><\/tr><tr><td>3D key\u00adpoints + object pose<\/td><td>Hand-object recon\u00adstruc\u00adtion, han\u00addover<\/td><td>Mul\u00adti-view or RGB\u2011D plus object mod\u00adels<\/td><td>High\u00adest<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Mul\u00adti-view cap\u00adture also helps label\u00ading itself. Simon et al. showed that detec\u00adtions from sev\u00ader\u00adal views can be tri\u00adan\u00adgu\u00adlat\u00aded in 3D and repro\u00adject\u00aded to cre\u00adate bet\u00adter labels for occlud\u00aded joints, an approach they called mul\u00adti\u00adview boot\u00adstrap\u00adping.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Which hand-object interaction labels matter for robot learning?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The hand-object labels that mat\u00adter most for robot learn\u00ading are con\u00adtact (which hand touch\u00ades which object, and when), grasp type, the objec\u00adt\u2019s iden\u00adti\u00adty and state, and action seg\u00adments with start and end times. Togeth\u00ader they turn hand motion into a descrip\u00adtion of the task that a manip\u00adu\u00adla\u00adtion pol\u00adi\u00adcy can learn from.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1600\" height=\"800\" src=\"https:\/\/www.graveiensai.com\/blog\/wp-content\/uploads\/2026\/10\/hand-pose-annotation__01-label-layers-v2.webp\" alt=\"Four stacked label layers for hand-object data: keypoints, contact, grasp type and object state.\" class=\"wp-image-273\" srcset=\"https:\/\/www.graveiensai.com\/blog\/wp-content\/uploads\/2026\/10\/hand-pose-annotation__01-label-layers-v2.webp 1600w, https:\/\/www.graveiensai.com\/blog\/wp-content\/uploads\/2026\/10\/hand-pose-annotation__01-label-layers-v2-300x150.webp 300w, https:\/\/www.graveiensai.com\/blog\/wp-content\/uploads\/2026\/10\/hand-pose-annotation__01-label-layers-v2-1024x512.webp 1024w, https:\/\/www.graveiensai.com\/blog\/wp-content\/uploads\/2026\/10\/hand-pose-annotation__01-label-layers-v2-768x384.webp 768w, https:\/\/www.graveiensai.com\/blog\/wp-content\/uploads\/2026\/10\/hand-pose-annotation__01-label-layers-v2-1536x768.webp 1536w\" sizes=\"auto, (max-width: 1600px) 100vw, 1600px\"><figcaption class=\"wp-element-caption\">Four label lay\u00aders that turn hand motion into task data.<\/figcaption><\/figure>\n\n\n\n<ul class=\"wp-block-list\">\n<li><b>Con\u00adtact.<\/b> Per-frame or per-seg\u00adment flags for hand-object con\u00adtact, ide\u00adal\u00adly per hand.<\/li>\n\n\n\n<li><b>Grasp type.<\/b> A short, fixed list (for exam\u00adple pow\u00ader, pre\u00adci\u00adsion, pinch) that your team can apply con\u00adsis\u00adtent\u00adly.<\/li>\n\n\n\n<li><b>Object iden\u00adti\u00adty and state.<\/b> Which object, and whether it is open, closed, full, emp\u00adty, cut or assem\u00adbled.<\/li>\n\n\n\n<li><b>Action seg\u00adments.<\/b> Start and end times for steps such as reach, grasp, move, place and release.<\/li>\n\n\n\n<li><b>Object pose (option\u00adal).<\/b> 6D pose when the pol\u00adi\u00adcy needs exact geom\u00ade\u00adtry.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Pub\u00adlic datasets show how these lay\u00aders com\u00adbine. DexY\u00adCB bench\u00admarks 2D key\u00adpoint detec\u00adtion, 6D object pose and 3D hand pose for hands grasp\u00ading objects, and adds a human-to-robot han\u00addover task. HOI4D pro\u00advides ego\u00adcen\u00adtric RGB\u2011D sequences with frame-wise 3D hand pose, object pose, seg\u00admen\u00adta\u00adtion and hand action labels. Pub\u00adlic data is a good ref\u00ader\u00adence; com\u00admer\u00adcial robots usu\u00adal\u00adly need cus\u00adtom cap\u00adture of their own tasks and set\u00adtings.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Label taxonomy examples by robot task<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">A tax\u00adon\u00ado\u00admy should be small enough to label con\u00adsis\u00adtent\u00adly and spe\u00adcif\u00adic enough to teach the task. The exam\u00adples below show how the same four lay\u00aders change with the job the robot will do.<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><thead><tr><th>Task fam\u00adi\u00adly<\/th><th>Grasp types to use<\/th><th>Object states to track<\/th><th>Action seg\u00adments<\/th><\/tr><\/thead><tbody><tr><td>Kitchen prepa\u00adra\u00adtion<\/td><td>Pow\u00ader, pre\u00adci\u00adsion, pinch, tool grip<\/td><td>Whole, cut, peeled, open, closed<\/td><td>Reach, grasp, cut, trans\u00adfer, release<\/td><\/tr><tr><td>Small-part assem\u00adbly<\/td><td>Pinch, tri\u00adpod, pre\u00adci\u00adsion<\/td><td>Loose, insert\u00aded, fas\u00adtened<\/td><td>Pick, align, insert, fas\u00adten, inspect<\/td><\/tr><tr><td>Ware\u00adhouse pick\u00ading<\/td><td>Pow\u00ader, wrap, two-hand lift<\/td><td>On shelf, in hand, in tote<\/td><td>Reach, grasp, lift, place, release<\/td><\/tr><tr><td>House\u00adhold tidy\u00ading<\/td><td>Pow\u00ader, pinch, flat-hand push<\/td><td>Fold\u00aded, unfold\u00aded, open, closed<\/td><td>Pick, fold, push, place, close<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Start with the small\u00adest list that cov\u00aders your tar\u00adget tasks, and add a label only when the pilot shows anno\u00adta\u00adtors need it. Every new class adds train\u00ading time for anno\u00adta\u00adtors and dis\u00adagree\u00adment in QA.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">The GRIP label spec<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Write a one-page spec before label\u00ading. We use four head\u00adings, GRIP, to keep it short and com\u00adplete.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1600\" height=\"800\" src=\"https:\/\/www.graveiensai.com\/blog\/wp-content\/uploads\/2026\/10\/hand-pose-annotation__02-grip-spec-v2.webp\" alt=\"One-page GRIP label spec: geometry, occlusion rules, interaction labels and pass criteria.\" class=\"wp-image-274\" srcset=\"https:\/\/www.graveiensai.com\/blog\/wp-content\/uploads\/2026\/10\/hand-pose-annotation__02-grip-spec-v2.webp 1600w, https:\/\/www.graveiensai.com\/blog\/wp-content\/uploads\/2026\/10\/hand-pose-annotation__02-grip-spec-v2-300x150.webp 300w, https:\/\/www.graveiensai.com\/blog\/wp-content\/uploads\/2026\/10\/hand-pose-annotation__02-grip-spec-v2-1024x512.webp 1024w, https:\/\/www.graveiensai.com\/blog\/wp-content\/uploads\/2026\/10\/hand-pose-annotation__02-grip-spec-v2-768x384.webp 768w, https:\/\/www.graveiensai.com\/blog\/wp-content\/uploads\/2026\/10\/hand-pose-annotation__02-grip-spec-v2-1536x768.webp 1536w\" sizes=\"auto, (max-width: 1600px) 100vw, 1600px\"><figcaption class=\"wp-element-caption\">The GRIP label spec fits on one page.<\/figcaption><\/figure>\n\n\n\n<ul class=\"wp-block-list\">\n<li><b>G: Geom\u00ade\u00adtry.<\/b> Key\u00adpoint lay\u00adout, joint order, 2D or 3D, coor\u00addi\u00adnate frames and units.<\/li>\n\n\n\n<li><b>R: Rules for occlu\u00adsion.<\/b> How to label joints that are hid\u00adden, out of frame or blurred: esti\u00admate, mark as not vis\u00adi\u00adble, or skip.<\/li>\n\n\n\n<li><b>I: Inter\u00adac\u00adtion labels.<\/b> Con\u00adtact, grasp types, object states and action seg\u00adments, with def\u00adi\u00adn\u00adi\u00adtions and exam\u00adples.<\/li>\n\n\n\n<li><b>P: Pass cri\u00adte\u00adria.<\/b> QA thresh\u00adolds, sam\u00adpling rate for review and what trig\u00adgers rework.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><b>Need a start\u00ading tem\u00adplate?<\/b> <a href=\"https:\/\/www.graveiensai.com\/contact-us\">Ask us for the hand-label spec tem\u00adplate<\/a> our team uses on scop\u00ading calls.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Which tools and formats are used for hand pose annotation?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Hand pose anno\u00adta\u00adtion is done in key\u00adpoint label\u00ading tools that sup\u00adport skele\u00adton tem\u00adplates, video inter\u00adpo\u00adla\u00adtion and, for 3D work, mul\u00adti-view or point-cloud views. Out\u00adput is usu\u00adal\u00adly COCO-style JSON key\u00adpoints with vis\u00adi\u00adbil\u00adi\u00adty flags for 2D, and per-frame 3D joint coor\u00addi\u00adnates or hand-mod\u00adel para\u00adme\u00adters for 3D. Agree the for\u00admat with your train\u00ading code before label\u00ading.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><b>Skele\u00adton tem\u00adplates.<\/b> The tool should enforce the joint order and con\u00adnec\u00adtions from your spec, so anno\u00adta\u00adtors can\u00adnot swap fin\u00adgers.<\/li>\n\n\n\n<li><b>Inter\u00adpo\u00adla\u00adtion and track\u00ading.<\/b> Label\u00ading keyframes and inter\u00adpo\u00adlat\u00ading between them saves time on slow motion; fast fin\u00adger motion still needs dense frames.<\/li>\n\n\n\n<li><b>Mul\u00adti-view sup\u00adport.<\/b> For 3D, anno\u00adta\u00adtors should see syn\u00adchro\u00adnized views side by side, with repro\u00adject\u00aded points to check con\u00adsis\u00adten\u00adcy.<\/li>\n\n\n\n<li><b>Attribute fields.<\/b> Con\u00adtact, grasp type and object state should be fields on the same time\u00adline, not a sep\u00ada\u00adrate project.<\/li>\n\n\n\n<li><b>Export.<\/b> Test a small export in your data loader first: joint order, image coor\u00addi\u00adnates vs nor\u00admal\u00adized val\u00adues, and frame index\u00ading.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">How does a hand pose annotation project run?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">A hand pose anno\u00adta\u00adtion project runs in five steps: write the label spec, label a pilot batch, cal\u00adi\u00adbrate anno\u00adta\u00adtors against gold exam\u00adples, label at scale with lay\u00adered review, then deliv\u00ader in your train\u00ading for\u00admat with a QA report. Most rework comes from vague occlu\u00adsion rules, so test them on hard clips dur\u00ading the pilot.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1600\" height=\"720\" src=\"https:\/\/www.graveiensai.com\/blog\/wp-content\/uploads\/2026\/10\/hand-pose-annotation__03-workflow-v2.webp\" alt=\"Five-step hand pose annotation workflow: spec, pilot batch, calibrate, label and review, deliver.\" class=\"wp-image-275\" srcset=\"https:\/\/www.graveiensai.com\/blog\/wp-content\/uploads\/2026\/10\/hand-pose-annotation__03-workflow-v2.webp 1600w, https:\/\/www.graveiensai.com\/blog\/wp-content\/uploads\/2026\/10\/hand-pose-annotation__03-workflow-v2-300x135.webp 300w, https:\/\/www.graveiensai.com\/blog\/wp-content\/uploads\/2026\/10\/hand-pose-annotation__03-workflow-v2-1024x461.webp 1024w, https:\/\/www.graveiensai.com\/blog\/wp-content\/uploads\/2026\/10\/hand-pose-annotation__03-workflow-v2-768x346.webp 768w, https:\/\/www.graveiensai.com\/blog\/wp-content\/uploads\/2026\/10\/hand-pose-annotation__03-workflow-v2-1536x691.webp 1536w\" sizes=\"auto, (max-width: 1600px) 100vw, 1600px\"><figcaption class=\"wp-element-caption\">A five-step work\u00adflow for hand pose anno\u00adta\u00adtion projects.<\/figcaption><\/figure>\n\n\n\n<ol class=\"wp-block-list\">\n<li><b>Spec.<\/b> Agree the GRIP spec and the deliv\u00adery for\u00admat.<\/li>\n\n\n\n<li><b>Pilot batch.<\/b> Label a small set that includes hard cas\u00ades: occlu\u00adsion, fast motion, both hands busy.<\/li>\n\n\n\n<li><b>Cal\u00adi\u00adbrate.<\/b> Build gold clips, mea\u00adsure agree\u00adment between anno\u00adta\u00adtors and tight\u00aden the guide\u00adlines.<\/li>\n\n\n\n<li><b>Label and review.<\/b> Label at scale with a review\u00ader pass and spot audits by a lead.<\/li>\n\n\n\n<li><b>Deliv\u00ader.<\/b> Export key\u00adpoints and inter\u00adac\u00adtion labels in your schema (for exam\u00adple, COCO-style JSON key\u00adpoints or a cus\u00adtom per-frame for\u00admat) with a QA sum\u00adma\u00adry.<\/li>\n<\/ol>\n\n\n\n<h3 class=\"wp-block-heading\">Bimanual and multi-person scenes<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Two hands work\u00ading togeth\u00ader, or two peo\u00adple in frame, cre\u00adate their own label\u00ading prob\u00adlems. Fix a left\/right con\u00adven\u00adtion from the cam\u00adera wear\u00ader\u2019s point of view, give each hand a per\u00adsis\u00adtent track ID, and decide in the spec whether hands that do not belong to the cam\u00adera wear\u00ader are labeled, skipped or blurred. Record the rule in the GRIP spec so every anno\u00adta\u00adtor applies it the same way.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">How do you measure hand pose annotation quality?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Mea\u00adsure hand pose anno\u00adta\u00adtion qual\u00adi\u00adty by com\u00adpar\u00ading labels with gold exam\u00adples and with a sec\u00adond anno\u00adta\u00adtor: key\u00adpoint dis\u00adtance error in pix\u00adels or mil\u00adlime\u00adters, agree\u00adment on vis\u00adi\u00adbil\u00adi\u00adty flags, and agree\u00adment on con\u00adtact, grasp type and seg\u00adment bound\u00adaries. Track results per anno\u00adta\u00adtor and per clip type, so hard cas\u00ades such as occlu\u00adsion get their own thresh\u00adold.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><b>Key\u00adpoint error.<\/b> Mean dis\u00adtance between labeled and gold key\u00adpoints, report\u00aded per joint.<\/li>\n\n\n\n<li><b>Vis\u00adi\u00adbil\u00adi\u00adty agree\u00adment.<\/b> Whether anno\u00adta\u00adtors agree on which joints are hid\u00adden.<\/li>\n\n\n\n<li><b>Inter\u00adac\u00adtion agree\u00adment.<\/b> Agree\u00adment on con\u00adtact flags, grasp types and seg\u00adment start and end times.<\/li>\n\n\n\n<li><b>Tem\u00adpo\u00adral smooth\u00adness.<\/b> Jumps between frames that a real hand could not make usu\u00adal\u00adly sig\u00adnal errors.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">How much hand pose data does a robot learning project need?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">There is no stan\u00addard num\u00adber of labeled frames. The amount depends on how many tasks, objects and set\u00adtings the pol\u00adi\u00adcy must cov\u00ader, whether labels are 2D or 3D, and how much pub\u00adlic data you can reuse for pre\u00adtrain\u00ading. Plan in stages: a pilot batch to set\u00adtle the spec, a first train\u00ading set sized to show whether the labels help, then growth dri\u00adven by mea\u00adsured gaps.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Cov\u00ader\u00adage usu\u00adal\u00adly mat\u00adters more than raw vol\u00adume. A few thou\u00adsand frames spread across many hands, objects, light\u00ading con\u00addi\u00adtions and cam\u00adera angles tend to teach more than many more frames from one par\u00adtic\u00adi\u00adpant at one table. Track cov\u00ader\u00adage in meta\u00adda\u00adta (task, object, set\u00adting, par\u00adtic\u00adi\u00adpant and cam\u00adera) so you can see where the next batch should go, and keep a fixed test set that is nev\u00ader used for train\u00ading.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Capture tips that make hand labeling easier<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li><b>Keep hands in frame.<\/b> Set head-mount\u00aded cam\u00adera angle and height so both hands stay vis\u00adi\u00adble dur\u00ading the task.<\/li>\n\n\n\n<li><b>Light the work area.<\/b> Low light caus\u00ades motion blur on fin\u00adgers, the hard\u00adest joints to label.<\/li>\n\n\n\n<li><b>Use a high\u00ader frame rate for fast tasks.<\/b> Fast fin\u00adger motion between frames makes inter\u00adpo\u00adla\u00adtion unre\u00adli\u00adable.<\/li>\n\n\n\n<li><b>Cal\u00adi\u00adbrate and sync cam\u00aderas.<\/b> 3D labels depend on intrin\u00adsics, extrin\u00adsics and a shared clock.<\/li>\n\n\n\n<li><b>Script the tasks.<\/b> Clear task steps make action seg\u00adments and con\u00adtact labels eas\u00adi\u00ader to define.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">For rigs, cal\u00adi\u00adbra\u00adtion files and con\u00adsent, see our guide to <a href=\"https:\/\/www.graveiensai.com\/blog\/egocentric-data-collection-vendor\/\">choos\u00ading an ego\u00adcen\u00adtric data col\u00adlec\u00adtion ven\u00addor<\/a>. For oth\u00ader label types such as box\u00ades and track\u00ading, see our <a href=\"https:\/\/www.graveiensai.com\/blog\/video-annotation\/\">video anno\u00adta\u00adtion guide<\/a>, and for where hand data fits in a robot data mix, our <a href=\"https:\/\/www.graveiensai.com\/blog\/humanoid-robot-training-data\/\">humanoid robot train\u00ading data guide<\/a>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">How Graveiens AI approaches hand pose annotation<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Graveiens AI\u2019s <a href=\"https:\/\/www.graveiensai.com\/data-annotation\">data anno\u00adta\u00adtion<\/a> team labels hand key\u00adpoints, con\u00adtact, grasp types, object states and action seg\u00adments to the clien\u00adt\u2019s spec, with lay\u00adered review and QA against agreed pass cri\u00adte\u00adria. When a pro\u00adgram also needs footage, we cap\u00adture it through con\u00adsent-backed <a href=\"https:\/\/www.graveiensai.com\/egocentric-video-data-collection\">ego\u00adcen\u00adtric video data col\u00adlec\u00adtion<\/a> and keep cap\u00adture and label\u00ading under one QA process.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Every pro\u00adgram fol\u00adlows our explic\u00adit-con\u00adsent work\u00adflow (par\u00adtic\u00adi\u00adpant onboard\u00ading, cap\u00adture man\u00adage\u00adment, meta\u00adda\u00adta tag\u00adging, QA), and clients are invoiced only for approved deliv\u00ader\u00adables. Graveiens AI is ISO 9001 cer\u00adti\u00adfied.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Frequently asked questions<\/h2>\n\n\n<div id=\"rank-math-faq\" class=\"rank-math-block\">\n<div class=\"rank-math-list \">\n<div id=\"faq-question-1761500000000\" class=\"rank-math-list-item\">\n<h3 class=\"rank-math-question \">What is the difference between hand pose annotation and hand tracking?<\/h3>\n<div class=\"rank-math-answer \">\n\n<p>Hand pose anno\u00adta\u00adtion is the human label\u00ading of joint posi\u00adtions in frames to cre\u00adate train\u00ading and test data. Hand track\u00ading is what a trained mod\u00adel or device does at run time, esti\u00admat\u00ading those posi\u00adtions auto\u00admat\u00adi\u00adcal\u00adly. Anno\u00adtat\u00aded data is used to train and eval\u00adu\u00adate track\u00aders, and to check track\u00ader out\u00adput before it is used as labels.<\/p>\n\n<\/div>\n<\/div>\n<div id=\"faq-question-1761500000001\" class=\"rank-math-list-item\">\n<h3 class=\"rank-math-question \">How many keypoints are used for a hand?<\/h3>\n<div class=\"rank-math-answer \">\n\n<p>Most projects use a lay\u00adout with the wrist plus sev\u00ader\u00adal joints along each fin\u00adger, often 21 points in total. What mat\u00adters most is con\u00adsis\u00adten\u00adcy: keep the same lay\u00adout, joint order and names across the dataset, and doc\u00adu\u00adment them in the label spec.<\/p>\n\n<\/div>\n<\/div>\n<div id=\"faq-question-1761500000002\" class=\"rank-math-list-item\">\n<h3 class=\"rank-math-question \">Can hand pose be annotated in egocentric video?<\/h3>\n<div class=\"rank-math-answer \">\n\n<p>Yes. Ego\u00adcen\u00adtric video shows hands close up dur\u00ading real tasks, which suits manip\u00adu\u00adla\u00adtion research, but hands often leave the frame or hide each oth\u00ader. Clear occlu\u00adsion rules, vis\u00adi\u00adbil\u00adi\u00adty flags and a cam\u00adera angle that keeps both hands in view make ego\u00adcen\u00adtric hand label\u00ading far more reli\u00adable.<\/p>\n\n<\/div>\n<\/div>\n<div id=\"faq-question-1761500000003\" class=\"rank-math-list-item\">\n<h3 class=\"rank-math-question \">Do I need 3D hand keypoints for robot learning?<\/h3>\n<div class=\"rank-math-answer \">\n\n<p>Not always. Many poli\u00adcies learn from 2D key\u00adpoints plus con\u00adtact and action labels. 3D key\u00adpoints help when the robot must match grasp geom\u00ade\u00adtry or retar\u00adget human motion to a robot hand, and they usu\u00adal\u00adly need cal\u00adi\u00adbrat\u00aded mul\u00adti-view or depth cap\u00adture.<\/p>\n\n<\/div>\n<\/div>\n<div id=\"faq-question-1761500000004\" class=\"rank-math-list-item\">\n<h3 class=\"rank-math-question \">What is a hand-object interaction dataset?<\/h3>\n<div class=\"rank-math-answer \">\n\n<p>A hand-object inter\u00adac\u00adtion dataset pairs hand pose with labels about the objects being han\u00addled: con\u00adtact, grasp type, object iden\u00adti\u00adty, state and some\u00adtimes 6D object pose. Pub\u00adlic exam\u00adples include DexY\u00adCB and HOI4D; com\u00admer\u00adcial teams often com\u00admis\u00adsion cus\u00adtom data for their own tasks and set\u00adtings.<\/p>\n\n<\/div>\n<\/div>\n<div id=\"faq-question-1761500000005\" class=\"rank-math-list-item\">\n<h3 class=\"rank-math-question \">Can AI pre-labels speed up hand pose annotation?<\/h3>\n<div class=\"rank-math-answer \">\n\n<p>Yes, a mod\u00adel can pro\u00adpose key\u00adpoints for anno\u00adta\u00adtors to cor\u00adrect, which often speeds up easy frames. Pre-labels are weak\u00adest on occlud\u00aded or fast-mov\u00ading fin\u00adgers, so keep full human review there and mea\u00adsure qual\u00adi\u00adty against gold clips rather than trust\u00ading mod\u00adel out\u00adput.<\/p>\n\n<\/div>\n<\/div>\n<div id=\"faq-question-1761500000006\" class=\"rank-math-list-item\">\n<h3 class=\"rank-math-question \">Which file format should hand pose labels be delivered in?<\/h3>\n<div class=\"rank-math-answer \">\n\n<p>For 2D work, COCO-style JSON key\u00adpoints with vis\u00adi\u00adbil\u00adi\u00adty flags are com\u00admon because many train\u00ading tools read them. For 3D work, per-frame joint coor\u00addi\u00adnates with cam\u00adera para\u00adme\u00adters, or hand-mod\u00adel para\u00adme\u00adters, are typ\u00adi\u00adcal. Test a small export with your own data loader before label\u00ading at scale.<\/p>\n\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n\n\n<h2 class=\"wp-block-heading\">Scope your hand annotation pilot<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Send us a few sam\u00adple clips and your tar\u00adget labels, and we will return a spec and a pilot plan. <a href=\"https:\/\/www.graveiensai.com\/data-annotation\">Explore data anno\u00adta\u00adtion ser\u00advices<\/a> or <a href=\"https:\/\/www.graveiensai.com\/contact-us\">scope a pilot billed only on approved files<\/a>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Sources<\/h2>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Simon, T., Joo, H., Matthews, I., Sheikh, Y. \u201cHand Key\u00adpoint Detec\u00adtion in Sin\u00adgle Images using Mul\u00adti\u00adview Boot\u00adstrap\u00adping.\u201d arXiv:1704.07809. <a href=\"https:\/\/arxiv.org\/abs\/1704.07809\" target=\"_blank\" rel=\"noopener\">arxiv.org\/abs\/1704.07809<\/a><\/li>\n\n\n\n<li>Chao, Y.-W. et al. \u201cDexY\u00adCB: A Bench\u00admark for Cap\u00adtur\u00ading Hand Grasp\u00ading of Objects.\u201d arXiv:2104.04631. <a href=\"https:\/\/arxiv.org\/abs\/2104.04631\" target=\"_blank\" rel=\"noopener\">arxiv.org\/abs\/2104.04631<\/a><\/li>\n\n\n\n<li>Liu, Y. et al. \u201cHOI4D: A 4D Ego\u00adcen\u00adtric Dataset for Cat\u00ade\u00adgo\u00adry-Lev\u00adel Human-Object Inter\u00adac\u00adtion.\u201d arXiv:2203.01577. <a href=\"https:\/\/arxiv.org\/abs\/2203.01577\" target=\"_blank\" rel=\"noopener\">arxiv.org\/abs\/2203.01577<\/a><\/li>\n<\/ol>\n\n\n\n<h2 class=\"wp-block-heading\">About the author<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Graveiens AI Edi\u00adto\u00adr\u00adi\u00adal Team. The team writes from day-to-day work run\u00adning con\u00adsent-backed data pro\u00adgrams for AI and robot\u00adics teams, includ\u00ading first-per\u00adson video cap\u00adture, hand and action anno\u00adta\u00adtion, and mul\u00adti\u00adlin\u00adgual voice data.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Hand pose anno\u00adta\u00adtion for robot learn\u00ading: 2D vs 3D key\u00adpoints, hand-object con\u00adtact and grasp labels, QA checks and cap\u00adture tips. Request a sam\u00adple set.<\/p>\n","protected":false},"author":1,"featured_media":272,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"wp_typography_post_enhancements_disabled":false,"footnotes":""},"categories":[4],"tags":[5,28,29,30],"class_list":["post-258","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-robotics-physical-ai","tag-egocentric-video","tag-hand-pose-annotation","tag-hand-object-interaction","tag-robot-learning"],"_links":{"self":[{"href":"https:\/\/www.graveiensai.com\/blog\/wp-json\/wp\/v2\/posts\/258","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.graveiensai.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.graveiensai.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.graveiensai.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.graveiensai.com\/blog\/wp-json\/wp\/v2\/comments?post=258"}],"version-history":[{"count":2,"href":"https:\/\/www.graveiensai.com\/blog\/wp-json\/wp\/v2\/posts\/258\/revisions"}],"predecessor-version":[{"id":276,"href":"https:\/\/www.graveiensai.com\/blog\/wp-json\/wp\/v2\/posts\/258\/revisions\/276"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.graveiensai.com\/blog\/wp-json\/wp\/v2\/media\/272"}],"wp:attachment":[{"href":"https:\/\/www.graveiensai.com\/blog\/wp-json\/wp\/v2\/media?parent=258"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.graveiensai.com\/blog\/wp-json\/wp\/v2\/categories?post=258"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.graveiensai.com\/blog\/wp-json\/wp\/v2\/tags?post=258"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}