2025-9-5

发现Gemini真的会按照我思考的顺序得到研究报告。

基本上我一步步看下来得到的疑问，它都会在下一步帮我解答。

annotations列表中的每个元素对应一张图片的标注。

segments_info是核心，它为mask.png中的每一个id（即每一个实例）提供了详细的语义信息，包括它属于哪个category_id，以及我们自定义的attributes字典。

{
    "images": [
        {"id": 1, "file_name": "image001.jpg", "width": 800, "height": 600}
    ],
    "categories": [... ], // 从第一步复制过来
    "annotations": [
        {
            "image_id": 1,
            "file_name": "panoptic_mask_001.png", // 指向第二步生成的掩码文件
            "segments_info": [
                {
                    "id": 1001, // 对应mask.png里的像素值
                    "category_id": 1, // 对应categories里的"rice"
                    "area": 150000,
                    "bbox": ,
                    "attributes": {
                        "dish_context": null,
                        "form": "grain",
                        "cooking_method": ["steamed"],
                        "is_staple": true
                    }
                },
                {
                    "id": 2001, // 对应mask.png里的像素值
                    "category_id": 2, // 对应categories里的"beef"
                    "area": 1200,
                    "bbox": ,
                    "attributes": {
                        "dish_context": "beef_bowl",
                        "form": "slice",
                        "cooking_method": ["stir-fried"],
                        "is_staple": false
                    }
                },
                {
                    "id": 2002, // 另一片牛肉
                    "category_id": 2,
                    "area": 1350,
                    "bbox": ,
                    "attributes": {
                        "dish_context": "beef_bowl",
                        "form": "slice",
                        "cooking_method": ["stir-fried"],
                        "is_staple": false
                    }
                }
                //... 为mask.png中每一个唯一的实例ID都创建一个条目
            ]
        }
    ]
}

标注流程

改标注菜品为标注食材。假设图中具有菜A，菜B……

首先对图中菜A进行标注：假如菜A中的食材是可分割大块的（比如，炒花菜）那么对每一朵花菜进行一个个标注，然后标注的所有花菜组成整体为炒花菜。（可能也不是这样，而是将同一种食材划分成一个类别）

{
  "info": {
    "description": "中餐食物数据集标注，包含上面两张图片",
    "version": "1.0",
    "year": 2024,
    "contributor": "AI标注团队"
  },
  "licenses": [
    {
      "url": "http://creativecommons.org/licenses/by/2.0/",
      "id": 1,
      "name": "Attribution License"
    }
  ],
  "images": [
    {
      "id": 2024001,
      "width": 1024,
      "height": 1024,
      "file_name": "chicken_leg_with_cauliflower.jpg",
      "license": 1,
      "date_captured": "2024-01-01 12:00:00"
    },
    {
      "id": 2024003,
      "width": 1024,
      "height": 1024,
      "file_name": "mixed_platter_001.jpg",
      "license": 1,
      "date_captured": "2024-01-01 12:00:00"
    }
  ],
  "categories": [
    { "id": 1, "name": "rice", "supercategory": "staple", "isthing": 0 },
    { "id": 2, "name": "chicken_leg", "supercategory": "meat", "isthing": 1 },
    { "id": 3, "name": "cauliflower", "supercategory": "vegetable", "isthing": 1 },
    { "id": 4, "name": "cured_pork", "supercategory": "meat", "isthing": 1 },
    { "id": 5, "name": "lotus_root", "supercategory": "vegetable", "isthing": 1 },
    { "id": 6, "name": "chili_pepper", "supercategory": "vegetable", "isthing": 1 },
    { "id": 7, "name": "pork", "supercategory": "meat", "isthing": 1 },
    { "id": 8, "name": "fried_tofu_puff", "supercategory": "soy_product", "isthing": 1 }
  ],
  "annotations": [
    {
      "image_id": 2024001,
      "file_name": "panoptic_mask_2024001.png",
      "segments_info": [
        {
          "id": 101, "category_id": 1, "iscrowd": 0, "bbox": , "area": 350000,
          "attributes": { "dish_context": null, "is_staple": true, "cooking_method": ["steamed"], "form": "grain" }
        },
        {
          "id": 201, "category_id": 2, "iscrowd": 0, "bbox": , "area": 280000,
          "attributes": { "dish_context": "braised_chicken_leg", "is_staple": false, "cooking_method": ["braised"], "form": "whole" }
        },
        {
          "id": 301, "category_id": 3, "iscrowd": 0, "bbox": , "area": 45000,
          "attributes": { "dish_context": "stir_fried_cauliflower", "is_staple": false, "cooking_method": ["stir-fried"], "form": "floret" }
        },
        {
          "id": 302, "category_id": 3, "iscrowd": 0, "bbox": , "area": 38000,
          "attributes": { "dish_context": "stir_fried_cauliflower", "is_staple": false, "cooking_method": ["stir-fried"], "form": "floret" }
        }
      ]
    },
    {
      "image_id": 2024003,
      "file_name": "panoptic_mask_2024003.png",
      "segments_info": [
        {
          "id": 101, "category_id": 1, "iscrowd": 0, "bbox": , "area": 380000,
          "attributes": { "dish_context": null, "is_staple": true, "cooking_method": ["steamed"], "form": "grain" }
        },
        {
          "id": 301, "category_id": 3, "iscrowd": 0, "bbox": , "area": 48000,
          "attributes": { "dish_context": "stir_fried_cauliflower", "is_staple": false, "cooking_method": ["stir-fried"], "form": "floret" }
        },
        {
          "id": 801, "category_id": 8, "iscrowd": 0, "bbox": , "area": 21000,
          "attributes": { "dish_context": "stir_fried_cauliflower", "is_staple": false, "cooking_method": ["stir-fried", "deep-fried"], "form": "chunk" }
        },
        {
          "id": 401, "category_id": 4, "iscrowd": 0, "bbox": , "area": 9000,
          "attributes": { "dish_context": "stir_fried_cured_pork_with_lotus_root", "is_staple": false, "cooking_method": ["stir-fried"], "form": "dice" }
        },
        {
          "id": 501, "category_id": 5, "iscrowd": 0, "bbox": , "area": 13000,
          "attributes": { "dish_context": "stir_fried_cured_pork_with_lotus_root", "is_staple": false, "cooking_method": ["stir-fried"], "form": "slice" }
        },
        {
          "id": 601, "category_id": 6, "iscrowd": 0, "bbox": , "area": 5600,
          "attributes": { "dish_context": "stir_fried_cured_pork_with_lotus_root", "is_staple": false, "cooking_method": ["stir-fried"], "form": "chunk" }
        },
        {
          "id": 701, "category_id": 7, "iscrowd": 0, "bbox": , "area": 19500,
          "attributes": { "dish_context": "guo_bao_rou", "is_staple": false, "cooking_method": ["deep-fried", "stir-fried_with_sauce"], "form": "slice" }
        },
        {
          "id": 702, "category_id": 7, "iscrowd": 0, "bbox": , "area": 16800,
          "attributes": { "dish_context": "guo_bao_rou", "is_staple": false, "cooking_method": ["deep-fried", "stir-fried_with_sauce"], "form": "slice" }
        }
      ]
    }
  ]
}