1*d5c09012SAndroid Build Coastguard Worker// Copyright 2021 Google LLC 2*d5c09012SAndroid Build Coastguard Worker// 3*d5c09012SAndroid Build Coastguard Worker// Licensed under the Apache License, Version 2.0 (the "License"); 4*d5c09012SAndroid Build Coastguard Worker// you may not use this file except in compliance with the License. 5*d5c09012SAndroid Build Coastguard Worker// You may obtain a copy of the License at 6*d5c09012SAndroid Build Coastguard Worker// 7*d5c09012SAndroid Build Coastguard Worker// http://www.apache.org/licenses/LICENSE-2.0 8*d5c09012SAndroid Build Coastguard Worker// 9*d5c09012SAndroid Build Coastguard Worker// Unless required by applicable law or agreed to in writing, software 10*d5c09012SAndroid Build Coastguard Worker// distributed under the License is distributed on an "AS IS" BASIS, 11*d5c09012SAndroid Build Coastguard Worker// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12*d5c09012SAndroid Build Coastguard Worker// See the License for the specific language governing permissions and 13*d5c09012SAndroid Build Coastguard Worker// limitations under the License. 14*d5c09012SAndroid Build Coastguard Worker 15*d5c09012SAndroid Build Coastguard Workersyntax = "proto3"; 16*d5c09012SAndroid Build Coastguard Worker 17*d5c09012SAndroid Build Coastguard Workerpackage google.cloud.automl.v1; 18*d5c09012SAndroid Build Coastguard Worker 19*d5c09012SAndroid Build Coastguard Workerimport "google/api/field_behavior.proto"; 20*d5c09012SAndroid Build Coastguard Worker 21*d5c09012SAndroid Build Coastguard Workeroption csharp_namespace = "Google.Cloud.AutoML.V1"; 22*d5c09012SAndroid Build Coastguard Workeroption go_package = "cloud.google.com/go/automl/apiv1/automlpb;automlpb"; 23*d5c09012SAndroid Build Coastguard Workeroption java_multiple_files = true; 24*d5c09012SAndroid Build Coastguard Workeroption java_package = "com.google.cloud.automl.v1"; 25*d5c09012SAndroid Build Coastguard Workeroption php_namespace = "Google\\Cloud\\AutoMl\\V1"; 26*d5c09012SAndroid Build Coastguard Workeroption ruby_package = "Google::Cloud::AutoML::V1"; 27*d5c09012SAndroid Build Coastguard Worker 28*d5c09012SAndroid Build Coastguard Worker// Input configuration for [AutoMl.ImportData][google.cloud.automl.v1.AutoMl.ImportData] action. 29*d5c09012SAndroid Build Coastguard Worker// 30*d5c09012SAndroid Build Coastguard Worker// The format of input depends on dataset_metadata the Dataset into which 31*d5c09012SAndroid Build Coastguard Worker// the import is happening has. As input source the 32*d5c09012SAndroid Build Coastguard Worker// [gcs_source][google.cloud.automl.v1.InputConfig.gcs_source] 33*d5c09012SAndroid Build Coastguard Worker// is expected, unless specified otherwise. Additionally any input .CSV file 34*d5c09012SAndroid Build Coastguard Worker// by itself must be 100MB or smaller, unless specified otherwise. 35*d5c09012SAndroid Build Coastguard Worker// If an "example" file (that is, image, video etc.) with identical content 36*d5c09012SAndroid Build Coastguard Worker// (even if it had different `GCS_FILE_PATH`) is mentioned multiple times, then 37*d5c09012SAndroid Build Coastguard Worker// its label, bounding boxes etc. are appended. The same file should be always 38*d5c09012SAndroid Build Coastguard Worker// provided with the same `ML_USE` and `GCS_FILE_PATH`, if it is not, then 39*d5c09012SAndroid Build Coastguard Worker// these values are nondeterministically selected from the given ones. 40*d5c09012SAndroid Build Coastguard Worker// 41*d5c09012SAndroid Build Coastguard Worker// The formats are represented in EBNF with commas being literal and with 42*d5c09012SAndroid Build Coastguard Worker// non-terminal symbols defined near the end of this comment. The formats are: 43*d5c09012SAndroid Build Coastguard Worker// 44*d5c09012SAndroid Build Coastguard Worker// <h4>AutoML Vision</h4> 45*d5c09012SAndroid Build Coastguard Worker// 46*d5c09012SAndroid Build Coastguard Worker// 47*d5c09012SAndroid Build Coastguard Worker// <div class="ds-selector-tabs"><section><h5>Classification</h5> 48*d5c09012SAndroid Build Coastguard Worker// 49*d5c09012SAndroid Build Coastguard Worker// See [Preparing your training 50*d5c09012SAndroid Build Coastguard Worker// data](https://cloud.google.com/vision/automl/docs/prepare) for more 51*d5c09012SAndroid Build Coastguard Worker// information. 52*d5c09012SAndroid Build Coastguard Worker// 53*d5c09012SAndroid Build Coastguard Worker// CSV file(s) with each line in format: 54*d5c09012SAndroid Build Coastguard Worker// 55*d5c09012SAndroid Build Coastguard Worker// ML_USE,GCS_FILE_PATH,LABEL,LABEL,... 56*d5c09012SAndroid Build Coastguard Worker// 57*d5c09012SAndroid Build Coastguard Worker// * `ML_USE` - Identifies the data set that the current row (file) applies 58*d5c09012SAndroid Build Coastguard Worker// to. 59*d5c09012SAndroid Build Coastguard Worker// This value can be one of the following: 60*d5c09012SAndroid Build Coastguard Worker// * `TRAIN` - Rows in this file are used to train the model. 61*d5c09012SAndroid Build Coastguard Worker// * `TEST` - Rows in this file are used to test the model during training. 62*d5c09012SAndroid Build Coastguard Worker// * `UNASSIGNED` - Rows in this file are not categorized. They are 63*d5c09012SAndroid Build Coastguard Worker// Automatically divided into train and test data. 80% for training and 64*d5c09012SAndroid Build Coastguard Worker// 20% for testing. 65*d5c09012SAndroid Build Coastguard Worker// 66*d5c09012SAndroid Build Coastguard Worker// * `GCS_FILE_PATH` - The Google Cloud Storage location of an image of up to 67*d5c09012SAndroid Build Coastguard Worker// 30MB in size. Supported extensions: .JPEG, .GIF, .PNG, .WEBP, .BMP, 68*d5c09012SAndroid Build Coastguard Worker// .TIFF, .ICO. 69*d5c09012SAndroid Build Coastguard Worker// 70*d5c09012SAndroid Build Coastguard Worker// * `LABEL` - A label that identifies the object in the image. 71*d5c09012SAndroid Build Coastguard Worker// 72*d5c09012SAndroid Build Coastguard Worker// For the `MULTICLASS` classification type, at most one `LABEL` is allowed 73*d5c09012SAndroid Build Coastguard Worker// per image. If an image has not yet been labeled, then it should be 74*d5c09012SAndroid Build Coastguard Worker// mentioned just once with no `LABEL`. 75*d5c09012SAndroid Build Coastguard Worker// 76*d5c09012SAndroid Build Coastguard Worker// Some sample rows: 77*d5c09012SAndroid Build Coastguard Worker// 78*d5c09012SAndroid Build Coastguard Worker// TRAIN,gs://folder/image1.jpg,daisy 79*d5c09012SAndroid Build Coastguard Worker// TEST,gs://folder/image2.jpg,dandelion,tulip,rose 80*d5c09012SAndroid Build Coastguard Worker// UNASSIGNED,gs://folder/image3.jpg,daisy 81*d5c09012SAndroid Build Coastguard Worker// UNASSIGNED,gs://folder/image4.jpg 82*d5c09012SAndroid Build Coastguard Worker// 83*d5c09012SAndroid Build Coastguard Worker// 84*d5c09012SAndroid Build Coastguard Worker// </section><section><h5>Object Detection</h5> 85*d5c09012SAndroid Build Coastguard Worker// See [Preparing your training 86*d5c09012SAndroid Build Coastguard Worker// data](https://cloud.google.com/vision/automl/object-detection/docs/prepare) 87*d5c09012SAndroid Build Coastguard Worker// for more information. 88*d5c09012SAndroid Build Coastguard Worker// 89*d5c09012SAndroid Build Coastguard Worker// A CSV file(s) with each line in format: 90*d5c09012SAndroid Build Coastguard Worker// 91*d5c09012SAndroid Build Coastguard Worker// ML_USE,GCS_FILE_PATH,[LABEL],(BOUNDING_BOX | ,,,,,,,) 92*d5c09012SAndroid Build Coastguard Worker// 93*d5c09012SAndroid Build Coastguard Worker// * `ML_USE` - Identifies the data set that the current row (file) applies 94*d5c09012SAndroid Build Coastguard Worker// to. 95*d5c09012SAndroid Build Coastguard Worker// This value can be one of the following: 96*d5c09012SAndroid Build Coastguard Worker// * `TRAIN` - Rows in this file are used to train the model. 97*d5c09012SAndroid Build Coastguard Worker// * `TEST` - Rows in this file are used to test the model during training. 98*d5c09012SAndroid Build Coastguard Worker// * `UNASSIGNED` - Rows in this file are not categorized. They are 99*d5c09012SAndroid Build Coastguard Worker// Automatically divided into train and test data. 80% for training and 100*d5c09012SAndroid Build Coastguard Worker// 20% for testing. 101*d5c09012SAndroid Build Coastguard Worker// 102*d5c09012SAndroid Build Coastguard Worker// * `GCS_FILE_PATH` - The Google Cloud Storage location of an image of up to 103*d5c09012SAndroid Build Coastguard Worker// 30MB in size. Supported extensions: .JPEG, .GIF, .PNG. Each image 104*d5c09012SAndroid Build Coastguard Worker// is assumed to be exhaustively labeled. 105*d5c09012SAndroid Build Coastguard Worker// 106*d5c09012SAndroid Build Coastguard Worker// * `LABEL` - A label that identifies the object in the image specified by the 107*d5c09012SAndroid Build Coastguard Worker// `BOUNDING_BOX`. 108*d5c09012SAndroid Build Coastguard Worker// 109*d5c09012SAndroid Build Coastguard Worker// * `BOUNDING BOX` - The vertices of an object in the example image. 110*d5c09012SAndroid Build Coastguard Worker// The minimum allowed `BOUNDING_BOX` edge length is 0.01, and no more than 111*d5c09012SAndroid Build Coastguard Worker// 500 `BOUNDING_BOX` instances per image are allowed (one `BOUNDING_BOX` 112*d5c09012SAndroid Build Coastguard Worker// per line). If an image has no looked for objects then it should be 113*d5c09012SAndroid Build Coastguard Worker// mentioned just once with no LABEL and the ",,,,,,," in place of the 114*d5c09012SAndroid Build Coastguard Worker// `BOUNDING_BOX`. 115*d5c09012SAndroid Build Coastguard Worker// 116*d5c09012SAndroid Build Coastguard Worker// **Four sample rows:** 117*d5c09012SAndroid Build Coastguard Worker// 118*d5c09012SAndroid Build Coastguard Worker// TRAIN,gs://folder/image1.png,car,0.1,0.1,,,0.3,0.3,, 119*d5c09012SAndroid Build Coastguard Worker// TRAIN,gs://folder/image1.png,bike,.7,.6,,,.8,.9,, 120*d5c09012SAndroid Build Coastguard Worker// UNASSIGNED,gs://folder/im2.png,car,0.1,0.1,0.2,0.1,0.2,0.3,0.1,0.3 121*d5c09012SAndroid Build Coastguard Worker// TEST,gs://folder/im3.png,,,,,,,,, 122*d5c09012SAndroid Build Coastguard Worker// </section> 123*d5c09012SAndroid Build Coastguard Worker// </div> 124*d5c09012SAndroid Build Coastguard Worker// 125*d5c09012SAndroid Build Coastguard Worker// 126*d5c09012SAndroid Build Coastguard Worker// <h4>AutoML Video Intelligence</h4> 127*d5c09012SAndroid Build Coastguard Worker// 128*d5c09012SAndroid Build Coastguard Worker// 129*d5c09012SAndroid Build Coastguard Worker// <div class="ds-selector-tabs"><section><h5>Classification</h5> 130*d5c09012SAndroid Build Coastguard Worker// 131*d5c09012SAndroid Build Coastguard Worker// See [Preparing your training 132*d5c09012SAndroid Build Coastguard Worker// data](https://cloud.google.com/video-intelligence/automl/docs/prepare) for 133*d5c09012SAndroid Build Coastguard Worker// more information. 134*d5c09012SAndroid Build Coastguard Worker// 135*d5c09012SAndroid Build Coastguard Worker// CSV file(s) with each line in format: 136*d5c09012SAndroid Build Coastguard Worker// 137*d5c09012SAndroid Build Coastguard Worker// ML_USE,GCS_FILE_PATH 138*d5c09012SAndroid Build Coastguard Worker// 139*d5c09012SAndroid Build Coastguard Worker// For `ML_USE`, do not use `VALIDATE`. 140*d5c09012SAndroid Build Coastguard Worker// 141*d5c09012SAndroid Build Coastguard Worker// `GCS_FILE_PATH` is the path to another .csv file that describes training 142*d5c09012SAndroid Build Coastguard Worker// example for a given `ML_USE`, using the following row format: 143*d5c09012SAndroid Build Coastguard Worker// 144*d5c09012SAndroid Build Coastguard Worker// GCS_FILE_PATH,(LABEL,TIME_SEGMENT_START,TIME_SEGMENT_END | ,,) 145*d5c09012SAndroid Build Coastguard Worker// 146*d5c09012SAndroid Build Coastguard Worker// Here `GCS_FILE_PATH` leads to a video of up to 50GB in size and up 147*d5c09012SAndroid Build Coastguard Worker// to 3h duration. Supported extensions: .MOV, .MPEG4, .MP4, .AVI. 148*d5c09012SAndroid Build Coastguard Worker// 149*d5c09012SAndroid Build Coastguard Worker// `TIME_SEGMENT_START` and `TIME_SEGMENT_END` must be within the 150*d5c09012SAndroid Build Coastguard Worker// length of the video, and the end time must be after the start time. Any 151*d5c09012SAndroid Build Coastguard Worker// segment of a video which has one or more labels on it, is considered a 152*d5c09012SAndroid Build Coastguard Worker// hard negative for all other labels. Any segment with no labels on 153*d5c09012SAndroid Build Coastguard Worker// it is considered to be unknown. If a whole video is unknown, then 154*d5c09012SAndroid Build Coastguard Worker// it should be mentioned just once with ",," in place of `LABEL, 155*d5c09012SAndroid Build Coastguard Worker// TIME_SEGMENT_START,TIME_SEGMENT_END`. 156*d5c09012SAndroid Build Coastguard Worker// 157*d5c09012SAndroid Build Coastguard Worker// Sample top level CSV file: 158*d5c09012SAndroid Build Coastguard Worker// 159*d5c09012SAndroid Build Coastguard Worker// TRAIN,gs://folder/train_videos.csv 160*d5c09012SAndroid Build Coastguard Worker// TEST,gs://folder/test_videos.csv 161*d5c09012SAndroid Build Coastguard Worker// UNASSIGNED,gs://folder/other_videos.csv 162*d5c09012SAndroid Build Coastguard Worker// 163*d5c09012SAndroid Build Coastguard Worker// Sample rows of a CSV file for a particular ML_USE: 164*d5c09012SAndroid Build Coastguard Worker// 165*d5c09012SAndroid Build Coastguard Worker// gs://folder/video1.avi,car,120,180.000021 166*d5c09012SAndroid Build Coastguard Worker// gs://folder/video1.avi,bike,150,180.000021 167*d5c09012SAndroid Build Coastguard Worker// gs://folder/vid2.avi,car,0,60.5 168*d5c09012SAndroid Build Coastguard Worker// gs://folder/vid3.avi,,, 169*d5c09012SAndroid Build Coastguard Worker// 170*d5c09012SAndroid Build Coastguard Worker// 171*d5c09012SAndroid Build Coastguard Worker// 172*d5c09012SAndroid Build Coastguard Worker// </section><section><h5>Object Tracking</h5> 173*d5c09012SAndroid Build Coastguard Worker// 174*d5c09012SAndroid Build Coastguard Worker// See [Preparing your training 175*d5c09012SAndroid Build Coastguard Worker// data](/video-intelligence/automl/object-tracking/docs/prepare) for more 176*d5c09012SAndroid Build Coastguard Worker// information. 177*d5c09012SAndroid Build Coastguard Worker// 178*d5c09012SAndroid Build Coastguard Worker// CSV file(s) with each line in format: 179*d5c09012SAndroid Build Coastguard Worker// 180*d5c09012SAndroid Build Coastguard Worker// ML_USE,GCS_FILE_PATH 181*d5c09012SAndroid Build Coastguard Worker// 182*d5c09012SAndroid Build Coastguard Worker// For `ML_USE`, do not use `VALIDATE`. 183*d5c09012SAndroid Build Coastguard Worker// 184*d5c09012SAndroid Build Coastguard Worker// `GCS_FILE_PATH` is the path to another .csv file that describes training 185*d5c09012SAndroid Build Coastguard Worker// example for a given `ML_USE`, using the following row format: 186*d5c09012SAndroid Build Coastguard Worker// 187*d5c09012SAndroid Build Coastguard Worker// GCS_FILE_PATH,LABEL,[INSTANCE_ID],TIMESTAMP,BOUNDING_BOX 188*d5c09012SAndroid Build Coastguard Worker// 189*d5c09012SAndroid Build Coastguard Worker// or 190*d5c09012SAndroid Build Coastguard Worker// 191*d5c09012SAndroid Build Coastguard Worker// GCS_FILE_PATH,,,,,,,,,, 192*d5c09012SAndroid Build Coastguard Worker// 193*d5c09012SAndroid Build Coastguard Worker// Here `GCS_FILE_PATH` leads to a video of up to 50GB in size and up 194*d5c09012SAndroid Build Coastguard Worker// to 3h duration. Supported extensions: .MOV, .MPEG4, .MP4, .AVI. 195*d5c09012SAndroid Build Coastguard Worker// Providing `INSTANCE_ID`s can help to obtain a better model. When 196*d5c09012SAndroid Build Coastguard Worker// a specific labeled entity leaves the video frame, and shows up 197*d5c09012SAndroid Build Coastguard Worker// afterwards it is not required, albeit preferable, that the same 198*d5c09012SAndroid Build Coastguard Worker// `INSTANCE_ID` is given to it. 199*d5c09012SAndroid Build Coastguard Worker// 200*d5c09012SAndroid Build Coastguard Worker// `TIMESTAMP` must be within the length of the video, the 201*d5c09012SAndroid Build Coastguard Worker// `BOUNDING_BOX` is assumed to be drawn on the closest video's frame 202*d5c09012SAndroid Build Coastguard Worker// to the `TIMESTAMP`. Any mentioned by the `TIMESTAMP` frame is expected 203*d5c09012SAndroid Build Coastguard Worker// to be exhaustively labeled and no more than 500 `BOUNDING_BOX`-es per 204*d5c09012SAndroid Build Coastguard Worker// frame are allowed. If a whole video is unknown, then it should be 205*d5c09012SAndroid Build Coastguard Worker// mentioned just once with ",,,,,,,,,," in place of `LABEL, 206*d5c09012SAndroid Build Coastguard Worker// [INSTANCE_ID],TIMESTAMP,BOUNDING_BOX`. 207*d5c09012SAndroid Build Coastguard Worker// 208*d5c09012SAndroid Build Coastguard Worker// Sample top level CSV file: 209*d5c09012SAndroid Build Coastguard Worker// 210*d5c09012SAndroid Build Coastguard Worker// TRAIN,gs://folder/train_videos.csv 211*d5c09012SAndroid Build Coastguard Worker// TEST,gs://folder/test_videos.csv 212*d5c09012SAndroid Build Coastguard Worker// UNASSIGNED,gs://folder/other_videos.csv 213*d5c09012SAndroid Build Coastguard Worker// 214*d5c09012SAndroid Build Coastguard Worker// Seven sample rows of a CSV file for a particular ML_USE: 215*d5c09012SAndroid Build Coastguard Worker// 216*d5c09012SAndroid Build Coastguard Worker// gs://folder/video1.avi,car,1,12.10,0.8,0.8,0.9,0.8,0.9,0.9,0.8,0.9 217*d5c09012SAndroid Build Coastguard Worker// gs://folder/video1.avi,car,1,12.90,0.4,0.8,0.5,0.8,0.5,0.9,0.4,0.9 218*d5c09012SAndroid Build Coastguard Worker// gs://folder/video1.avi,car,2,12.10,.4,.2,.5,.2,.5,.3,.4,.3 219*d5c09012SAndroid Build Coastguard Worker// gs://folder/video1.avi,car,2,12.90,.8,.2,,,.9,.3,, 220*d5c09012SAndroid Build Coastguard Worker// gs://folder/video1.avi,bike,,12.50,.45,.45,,,.55,.55,, 221*d5c09012SAndroid Build Coastguard Worker// gs://folder/video2.avi,car,1,0,.1,.9,,,.9,.1,, 222*d5c09012SAndroid Build Coastguard Worker// gs://folder/video2.avi,,,,,,,,,,, 223*d5c09012SAndroid Build Coastguard Worker// </section> 224*d5c09012SAndroid Build Coastguard Worker// </div> 225*d5c09012SAndroid Build Coastguard Worker// 226*d5c09012SAndroid Build Coastguard Worker// 227*d5c09012SAndroid Build Coastguard Worker// <h4>AutoML Natural Language</h4> 228*d5c09012SAndroid Build Coastguard Worker// 229*d5c09012SAndroid Build Coastguard Worker// 230*d5c09012SAndroid Build Coastguard Worker// <div class="ds-selector-tabs"><section><h5>Entity Extraction</h5> 231*d5c09012SAndroid Build Coastguard Worker// 232*d5c09012SAndroid Build Coastguard Worker// See [Preparing your training 233*d5c09012SAndroid Build Coastguard Worker// data](/natural-language/automl/entity-analysis/docs/prepare) for more 234*d5c09012SAndroid Build Coastguard Worker// information. 235*d5c09012SAndroid Build Coastguard Worker// 236*d5c09012SAndroid Build Coastguard Worker// One or more CSV file(s) with each line in the following format: 237*d5c09012SAndroid Build Coastguard Worker// 238*d5c09012SAndroid Build Coastguard Worker// ML_USE,GCS_FILE_PATH 239*d5c09012SAndroid Build Coastguard Worker// 240*d5c09012SAndroid Build Coastguard Worker// * `ML_USE` - Identifies the data set that the current row (file) applies 241*d5c09012SAndroid Build Coastguard Worker// to. 242*d5c09012SAndroid Build Coastguard Worker// This value can be one of the following: 243*d5c09012SAndroid Build Coastguard Worker// * `TRAIN` - Rows in this file are used to train the model. 244*d5c09012SAndroid Build Coastguard Worker// * `TEST` - Rows in this file are used to test the model during training. 245*d5c09012SAndroid Build Coastguard Worker// * `UNASSIGNED` - Rows in this file are not categorized. They are 246*d5c09012SAndroid Build Coastguard Worker// Automatically divided into train and test data. 80% for training and 247*d5c09012SAndroid Build Coastguard Worker// 20% for testing.. 248*d5c09012SAndroid Build Coastguard Worker// 249*d5c09012SAndroid Build Coastguard Worker// * `GCS_FILE_PATH` - a Identifies JSON Lines (.JSONL) file stored in 250*d5c09012SAndroid Build Coastguard Worker// Google Cloud Storage that contains in-line text in-line as documents 251*d5c09012SAndroid Build Coastguard Worker// for model training. 252*d5c09012SAndroid Build Coastguard Worker// 253*d5c09012SAndroid Build Coastguard Worker// After the training data set has been determined from the `TRAIN` and 254*d5c09012SAndroid Build Coastguard Worker// `UNASSIGNED` CSV files, the training data is divided into train and 255*d5c09012SAndroid Build Coastguard Worker// validation data sets. 70% for training and 30% for validation. 256*d5c09012SAndroid Build Coastguard Worker// 257*d5c09012SAndroid Build Coastguard Worker// For example: 258*d5c09012SAndroid Build Coastguard Worker// 259*d5c09012SAndroid Build Coastguard Worker// TRAIN,gs://folder/file1.jsonl 260*d5c09012SAndroid Build Coastguard Worker// VALIDATE,gs://folder/file2.jsonl 261*d5c09012SAndroid Build Coastguard Worker// TEST,gs://folder/file3.jsonl 262*d5c09012SAndroid Build Coastguard Worker// 263*d5c09012SAndroid Build Coastguard Worker// **In-line JSONL files** 264*d5c09012SAndroid Build Coastguard Worker// 265*d5c09012SAndroid Build Coastguard Worker// In-line .JSONL files contain, per line, a JSON document that wraps a 266*d5c09012SAndroid Build Coastguard Worker// [`text_snippet`][google.cloud.automl.v1.TextSnippet] field followed by 267*d5c09012SAndroid Build Coastguard Worker// one or more [`annotations`][google.cloud.automl.v1.AnnotationPayload] 268*d5c09012SAndroid Build Coastguard Worker// fields, which have `display_name` and `text_extraction` fields to describe 269*d5c09012SAndroid Build Coastguard Worker// the entity from the text snippet. Multiple JSON documents can be separated 270*d5c09012SAndroid Build Coastguard Worker// using line breaks (\n). 271*d5c09012SAndroid Build Coastguard Worker// 272*d5c09012SAndroid Build Coastguard Worker// The supplied text must be annotated exhaustively. For example, if you 273*d5c09012SAndroid Build Coastguard Worker// include the text "horse", but do not label it as "animal", 274*d5c09012SAndroid Build Coastguard Worker// then "horse" is assumed to not be an "animal". 275*d5c09012SAndroid Build Coastguard Worker// 276*d5c09012SAndroid Build Coastguard Worker// Any given text snippet content must have 30,000 characters or 277*d5c09012SAndroid Build Coastguard Worker// less, and also be UTF-8 NFC encoded. ASCII is accepted as it is 278*d5c09012SAndroid Build Coastguard Worker// UTF-8 NFC encoded. 279*d5c09012SAndroid Build Coastguard Worker// 280*d5c09012SAndroid Build Coastguard Worker// For example: 281*d5c09012SAndroid Build Coastguard Worker// 282*d5c09012SAndroid Build Coastguard Worker// { 283*d5c09012SAndroid Build Coastguard Worker// "text_snippet": { 284*d5c09012SAndroid Build Coastguard Worker// "content": "dog car cat" 285*d5c09012SAndroid Build Coastguard Worker// }, 286*d5c09012SAndroid Build Coastguard Worker// "annotations": [ 287*d5c09012SAndroid Build Coastguard Worker// { 288*d5c09012SAndroid Build Coastguard Worker// "display_name": "animal", 289*d5c09012SAndroid Build Coastguard Worker// "text_extraction": { 290*d5c09012SAndroid Build Coastguard Worker// "text_segment": {"start_offset": 0, "end_offset": 2} 291*d5c09012SAndroid Build Coastguard Worker// } 292*d5c09012SAndroid Build Coastguard Worker// }, 293*d5c09012SAndroid Build Coastguard Worker// { 294*d5c09012SAndroid Build Coastguard Worker// "display_name": "vehicle", 295*d5c09012SAndroid Build Coastguard Worker// "text_extraction": { 296*d5c09012SAndroid Build Coastguard Worker// "text_segment": {"start_offset": 4, "end_offset": 6} 297*d5c09012SAndroid Build Coastguard Worker// } 298*d5c09012SAndroid Build Coastguard Worker// }, 299*d5c09012SAndroid Build Coastguard Worker// { 300*d5c09012SAndroid Build Coastguard Worker// "display_name": "animal", 301*d5c09012SAndroid Build Coastguard Worker// "text_extraction": { 302*d5c09012SAndroid Build Coastguard Worker// "text_segment": {"start_offset": 8, "end_offset": 10} 303*d5c09012SAndroid Build Coastguard Worker// } 304*d5c09012SAndroid Build Coastguard Worker// } 305*d5c09012SAndroid Build Coastguard Worker// ] 306*d5c09012SAndroid Build Coastguard Worker// }\n 307*d5c09012SAndroid Build Coastguard Worker// { 308*d5c09012SAndroid Build Coastguard Worker// "text_snippet": { 309*d5c09012SAndroid Build Coastguard Worker// "content": "This dog is good." 310*d5c09012SAndroid Build Coastguard Worker// }, 311*d5c09012SAndroid Build Coastguard Worker// "annotations": [ 312*d5c09012SAndroid Build Coastguard Worker// { 313*d5c09012SAndroid Build Coastguard Worker// "display_name": "animal", 314*d5c09012SAndroid Build Coastguard Worker// "text_extraction": { 315*d5c09012SAndroid Build Coastguard Worker// "text_segment": {"start_offset": 5, "end_offset": 7} 316*d5c09012SAndroid Build Coastguard Worker// } 317*d5c09012SAndroid Build Coastguard Worker// } 318*d5c09012SAndroid Build Coastguard Worker// ] 319*d5c09012SAndroid Build Coastguard Worker// } 320*d5c09012SAndroid Build Coastguard Worker// 321*d5c09012SAndroid Build Coastguard Worker// **JSONL files that reference documents** 322*d5c09012SAndroid Build Coastguard Worker// 323*d5c09012SAndroid Build Coastguard Worker// .JSONL files contain, per line, a JSON document that wraps a 324*d5c09012SAndroid Build Coastguard Worker// `input_config` that contains the path to a source document. 325*d5c09012SAndroid Build Coastguard Worker// Multiple JSON documents can be separated using line breaks (\n). 326*d5c09012SAndroid Build Coastguard Worker// 327*d5c09012SAndroid Build Coastguard Worker// Supported document extensions: .PDF, .TIF, .TIFF 328*d5c09012SAndroid Build Coastguard Worker// 329*d5c09012SAndroid Build Coastguard Worker// For example: 330*d5c09012SAndroid Build Coastguard Worker// 331*d5c09012SAndroid Build Coastguard Worker// { 332*d5c09012SAndroid Build Coastguard Worker// "document": { 333*d5c09012SAndroid Build Coastguard Worker// "input_config": { 334*d5c09012SAndroid Build Coastguard Worker// "gcs_source": { "input_uris": [ "gs://folder/document1.pdf" ] 335*d5c09012SAndroid Build Coastguard Worker// } 336*d5c09012SAndroid Build Coastguard Worker// } 337*d5c09012SAndroid Build Coastguard Worker// } 338*d5c09012SAndroid Build Coastguard Worker// }\n 339*d5c09012SAndroid Build Coastguard Worker// { 340*d5c09012SAndroid Build Coastguard Worker// "document": { 341*d5c09012SAndroid Build Coastguard Worker// "input_config": { 342*d5c09012SAndroid Build Coastguard Worker// "gcs_source": { "input_uris": [ "gs://folder/document2.tif" ] 343*d5c09012SAndroid Build Coastguard Worker// } 344*d5c09012SAndroid Build Coastguard Worker// } 345*d5c09012SAndroid Build Coastguard Worker// } 346*d5c09012SAndroid Build Coastguard Worker// } 347*d5c09012SAndroid Build Coastguard Worker// 348*d5c09012SAndroid Build Coastguard Worker// **In-line JSONL files with document layout information** 349*d5c09012SAndroid Build Coastguard Worker// 350*d5c09012SAndroid Build Coastguard Worker// **Note:** You can only annotate documents using the UI. The format described 351*d5c09012SAndroid Build Coastguard Worker// below applies to annotated documents exported using the UI or `exportData`. 352*d5c09012SAndroid Build Coastguard Worker// 353*d5c09012SAndroid Build Coastguard Worker// In-line .JSONL files for documents contain, per line, a JSON document 354*d5c09012SAndroid Build Coastguard Worker// that wraps a `document` field that provides the textual content of the 355*d5c09012SAndroid Build Coastguard Worker// document and the layout information. 356*d5c09012SAndroid Build Coastguard Worker// 357*d5c09012SAndroid Build Coastguard Worker// For example: 358*d5c09012SAndroid Build Coastguard Worker// 359*d5c09012SAndroid Build Coastguard Worker// { 360*d5c09012SAndroid Build Coastguard Worker// "document": { 361*d5c09012SAndroid Build Coastguard Worker// "document_text": { 362*d5c09012SAndroid Build Coastguard Worker// "content": "dog car cat" 363*d5c09012SAndroid Build Coastguard Worker// } 364*d5c09012SAndroid Build Coastguard Worker// "layout": [ 365*d5c09012SAndroid Build Coastguard Worker// { 366*d5c09012SAndroid Build Coastguard Worker// "text_segment": { 367*d5c09012SAndroid Build Coastguard Worker// "start_offset": 0, 368*d5c09012SAndroid Build Coastguard Worker// "end_offset": 11, 369*d5c09012SAndroid Build Coastguard Worker// }, 370*d5c09012SAndroid Build Coastguard Worker// "page_number": 1, 371*d5c09012SAndroid Build Coastguard Worker// "bounding_poly": { 372*d5c09012SAndroid Build Coastguard Worker// "normalized_vertices": [ 373*d5c09012SAndroid Build Coastguard Worker// {"x": 0.1, "y": 0.1}, 374*d5c09012SAndroid Build Coastguard Worker// {"x": 0.1, "y": 0.3}, 375*d5c09012SAndroid Build Coastguard Worker// {"x": 0.3, "y": 0.3}, 376*d5c09012SAndroid Build Coastguard Worker// {"x": 0.3, "y": 0.1}, 377*d5c09012SAndroid Build Coastguard Worker// ], 378*d5c09012SAndroid Build Coastguard Worker// }, 379*d5c09012SAndroid Build Coastguard Worker// "text_segment_type": TOKEN, 380*d5c09012SAndroid Build Coastguard Worker// } 381*d5c09012SAndroid Build Coastguard Worker// ], 382*d5c09012SAndroid Build Coastguard Worker// "document_dimensions": { 383*d5c09012SAndroid Build Coastguard Worker// "width": 8.27, 384*d5c09012SAndroid Build Coastguard Worker// "height": 11.69, 385*d5c09012SAndroid Build Coastguard Worker// "unit": INCH, 386*d5c09012SAndroid Build Coastguard Worker// } 387*d5c09012SAndroid Build Coastguard Worker// "page_count": 3, 388*d5c09012SAndroid Build Coastguard Worker// }, 389*d5c09012SAndroid Build Coastguard Worker// "annotations": [ 390*d5c09012SAndroid Build Coastguard Worker// { 391*d5c09012SAndroid Build Coastguard Worker// "display_name": "animal", 392*d5c09012SAndroid Build Coastguard Worker// "text_extraction": { 393*d5c09012SAndroid Build Coastguard Worker// "text_segment": {"start_offset": 0, "end_offset": 3} 394*d5c09012SAndroid Build Coastguard Worker// } 395*d5c09012SAndroid Build Coastguard Worker// }, 396*d5c09012SAndroid Build Coastguard Worker// { 397*d5c09012SAndroid Build Coastguard Worker// "display_name": "vehicle", 398*d5c09012SAndroid Build Coastguard Worker// "text_extraction": { 399*d5c09012SAndroid Build Coastguard Worker// "text_segment": {"start_offset": 4, "end_offset": 7} 400*d5c09012SAndroid Build Coastguard Worker// } 401*d5c09012SAndroid Build Coastguard Worker// }, 402*d5c09012SAndroid Build Coastguard Worker// { 403*d5c09012SAndroid Build Coastguard Worker// "display_name": "animal", 404*d5c09012SAndroid Build Coastguard Worker// "text_extraction": { 405*d5c09012SAndroid Build Coastguard Worker// "text_segment": {"start_offset": 8, "end_offset": 11} 406*d5c09012SAndroid Build Coastguard Worker// } 407*d5c09012SAndroid Build Coastguard Worker// }, 408*d5c09012SAndroid Build Coastguard Worker// ], 409*d5c09012SAndroid Build Coastguard Worker// 410*d5c09012SAndroid Build Coastguard Worker// 411*d5c09012SAndroid Build Coastguard Worker// 412*d5c09012SAndroid Build Coastguard Worker// 413*d5c09012SAndroid Build Coastguard Worker// </section><section><h5>Classification</h5> 414*d5c09012SAndroid Build Coastguard Worker// 415*d5c09012SAndroid Build Coastguard Worker// See [Preparing your training 416*d5c09012SAndroid Build Coastguard Worker// data](https://cloud.google.com/natural-language/automl/docs/prepare) for more 417*d5c09012SAndroid Build Coastguard Worker// information. 418*d5c09012SAndroid Build Coastguard Worker// 419*d5c09012SAndroid Build Coastguard Worker// One or more CSV file(s) with each line in the following format: 420*d5c09012SAndroid Build Coastguard Worker// 421*d5c09012SAndroid Build Coastguard Worker// ML_USE,(TEXT_SNIPPET | GCS_FILE_PATH),LABEL,LABEL,... 422*d5c09012SAndroid Build Coastguard Worker// 423*d5c09012SAndroid Build Coastguard Worker// * `ML_USE` - Identifies the data set that the current row (file) applies 424*d5c09012SAndroid Build Coastguard Worker// to. 425*d5c09012SAndroid Build Coastguard Worker// This value can be one of the following: 426*d5c09012SAndroid Build Coastguard Worker// * `TRAIN` - Rows in this file are used to train the model. 427*d5c09012SAndroid Build Coastguard Worker// * `TEST` - Rows in this file are used to test the model during training. 428*d5c09012SAndroid Build Coastguard Worker// * `UNASSIGNED` - Rows in this file are not categorized. They are 429*d5c09012SAndroid Build Coastguard Worker// Automatically divided into train and test data. 80% for training and 430*d5c09012SAndroid Build Coastguard Worker// 20% for testing. 431*d5c09012SAndroid Build Coastguard Worker// 432*d5c09012SAndroid Build Coastguard Worker// * `TEXT_SNIPPET` and `GCS_FILE_PATH` are distinguished by a pattern. If 433*d5c09012SAndroid Build Coastguard Worker// the column content is a valid Google Cloud Storage file path, that is, 434*d5c09012SAndroid Build Coastguard Worker// prefixed by "gs://", it is treated as a `GCS_FILE_PATH`. Otherwise, if 435*d5c09012SAndroid Build Coastguard Worker// the content is enclosed in double quotes (""), it is treated as a 436*d5c09012SAndroid Build Coastguard Worker// `TEXT_SNIPPET`. For `GCS_FILE_PATH`, the path must lead to a 437*d5c09012SAndroid Build Coastguard Worker// file with supported extension and UTF-8 encoding, for example, 438*d5c09012SAndroid Build Coastguard Worker// "gs://folder/content.txt" AutoML imports the file content 439*d5c09012SAndroid Build Coastguard Worker// as a text snippet. For `TEXT_SNIPPET`, AutoML imports the column content 440*d5c09012SAndroid Build Coastguard Worker// excluding quotes. In both cases, size of the content must be 10MB or 441*d5c09012SAndroid Build Coastguard Worker// less in size. For zip files, the size of each file inside the zip must be 442*d5c09012SAndroid Build Coastguard Worker// 10MB or less in size. 443*d5c09012SAndroid Build Coastguard Worker// 444*d5c09012SAndroid Build Coastguard Worker// For the `MULTICLASS` classification type, at most one `LABEL` is allowed. 445*d5c09012SAndroid Build Coastguard Worker// 446*d5c09012SAndroid Build Coastguard Worker// The `ML_USE` and `LABEL` columns are optional. 447*d5c09012SAndroid Build Coastguard Worker// Supported file extensions: .TXT, .PDF, .TIF, .TIFF, .ZIP 448*d5c09012SAndroid Build Coastguard Worker// 449*d5c09012SAndroid Build Coastguard Worker// A maximum of 100 unique labels are allowed per CSV row. 450*d5c09012SAndroid Build Coastguard Worker// 451*d5c09012SAndroid Build Coastguard Worker// Sample rows: 452*d5c09012SAndroid Build Coastguard Worker// 453*d5c09012SAndroid Build Coastguard Worker// TRAIN,"They have bad food and very rude",RudeService,BadFood 454*d5c09012SAndroid Build Coastguard Worker// gs://folder/content.txt,SlowService 455*d5c09012SAndroid Build Coastguard Worker// TEST,gs://folder/document.pdf 456*d5c09012SAndroid Build Coastguard Worker// VALIDATE,gs://folder/text_files.zip,BadFood 457*d5c09012SAndroid Build Coastguard Worker// 458*d5c09012SAndroid Build Coastguard Worker// 459*d5c09012SAndroid Build Coastguard Worker// 460*d5c09012SAndroid Build Coastguard Worker// </section><section><h5>Sentiment Analysis</h5> 461*d5c09012SAndroid Build Coastguard Worker// 462*d5c09012SAndroid Build Coastguard Worker// See [Preparing your training 463*d5c09012SAndroid Build Coastguard Worker// data](https://cloud.google.com/natural-language/automl/docs/prepare) for more 464*d5c09012SAndroid Build Coastguard Worker// information. 465*d5c09012SAndroid Build Coastguard Worker// 466*d5c09012SAndroid Build Coastguard Worker// CSV file(s) with each line in format: 467*d5c09012SAndroid Build Coastguard Worker// 468*d5c09012SAndroid Build Coastguard Worker// ML_USE,(TEXT_SNIPPET | GCS_FILE_PATH),SENTIMENT 469*d5c09012SAndroid Build Coastguard Worker// 470*d5c09012SAndroid Build Coastguard Worker// * `ML_USE` - Identifies the data set that the current row (file) applies 471*d5c09012SAndroid Build Coastguard Worker// to. 472*d5c09012SAndroid Build Coastguard Worker// This value can be one of the following: 473*d5c09012SAndroid Build Coastguard Worker// * `TRAIN` - Rows in this file are used to train the model. 474*d5c09012SAndroid Build Coastguard Worker// * `TEST` - Rows in this file are used to test the model during training. 475*d5c09012SAndroid Build Coastguard Worker// * `UNASSIGNED` - Rows in this file are not categorized. They are 476*d5c09012SAndroid Build Coastguard Worker// Automatically divided into train and test data. 80% for training and 477*d5c09012SAndroid Build Coastguard Worker// 20% for testing. 478*d5c09012SAndroid Build Coastguard Worker// 479*d5c09012SAndroid Build Coastguard Worker// * `TEXT_SNIPPET` and `GCS_FILE_PATH` are distinguished by a pattern. If 480*d5c09012SAndroid Build Coastguard Worker// the column content is a valid Google Cloud Storage file path, that is, 481*d5c09012SAndroid Build Coastguard Worker// prefixed by "gs://", it is treated as a `GCS_FILE_PATH`. Otherwise, if 482*d5c09012SAndroid Build Coastguard Worker// the content is enclosed in double quotes (""), it is treated as a 483*d5c09012SAndroid Build Coastguard Worker// `TEXT_SNIPPET`. For `GCS_FILE_PATH`, the path must lead to a 484*d5c09012SAndroid Build Coastguard Worker// file with supported extension and UTF-8 encoding, for example, 485*d5c09012SAndroid Build Coastguard Worker// "gs://folder/content.txt" AutoML imports the file content 486*d5c09012SAndroid Build Coastguard Worker// as a text snippet. For `TEXT_SNIPPET`, AutoML imports the column content 487*d5c09012SAndroid Build Coastguard Worker// excluding quotes. In both cases, size of the content must be 128kB or 488*d5c09012SAndroid Build Coastguard Worker// less in size. For zip files, the size of each file inside the zip must be 489*d5c09012SAndroid Build Coastguard Worker// 128kB or less in size. 490*d5c09012SAndroid Build Coastguard Worker// 491*d5c09012SAndroid Build Coastguard Worker// The `ML_USE` and `SENTIMENT` columns are optional. 492*d5c09012SAndroid Build Coastguard Worker// Supported file extensions: .TXT, .PDF, .TIF, .TIFF, .ZIP 493*d5c09012SAndroid Build Coastguard Worker// 494*d5c09012SAndroid Build Coastguard Worker// * `SENTIMENT` - An integer between 0 and 495*d5c09012SAndroid Build Coastguard Worker// Dataset.text_sentiment_dataset_metadata.sentiment_max 496*d5c09012SAndroid Build Coastguard Worker// (inclusive). Describes the ordinal of the sentiment - higher 497*d5c09012SAndroid Build Coastguard Worker// value means a more positive sentiment. All the values are 498*d5c09012SAndroid Build Coastguard Worker// completely relative, i.e. neither 0 needs to mean a negative or 499*d5c09012SAndroid Build Coastguard Worker// neutral sentiment nor sentiment_max needs to mean a positive one - 500*d5c09012SAndroid Build Coastguard Worker// it is just required that 0 is the least positive sentiment 501*d5c09012SAndroid Build Coastguard Worker// in the data, and sentiment_max is the most positive one. 502*d5c09012SAndroid Build Coastguard Worker// The SENTIMENT shouldn't be confused with "score" or "magnitude" 503*d5c09012SAndroid Build Coastguard Worker// from the previous Natural Language Sentiment Analysis API. 504*d5c09012SAndroid Build Coastguard Worker// All SENTIMENT values between 0 and sentiment_max must be 505*d5c09012SAndroid Build Coastguard Worker// represented in the imported data. On prediction the same 0 to 506*d5c09012SAndroid Build Coastguard Worker// sentiment_max range will be used. The difference between 507*d5c09012SAndroid Build Coastguard Worker// neighboring sentiment values needs not to be uniform, e.g. 1 and 508*d5c09012SAndroid Build Coastguard Worker// 2 may be similar whereas the difference between 2 and 3 may be 509*d5c09012SAndroid Build Coastguard Worker// large. 510*d5c09012SAndroid Build Coastguard Worker// 511*d5c09012SAndroid Build Coastguard Worker// Sample rows: 512*d5c09012SAndroid Build Coastguard Worker// 513*d5c09012SAndroid Build Coastguard Worker// TRAIN,"@freewrytin this is way too good for your product",2 514*d5c09012SAndroid Build Coastguard Worker// gs://folder/content.txt,3 515*d5c09012SAndroid Build Coastguard Worker// TEST,gs://folder/document.pdf 516*d5c09012SAndroid Build Coastguard Worker// VALIDATE,gs://folder/text_files.zip,2 517*d5c09012SAndroid Build Coastguard Worker// </section> 518*d5c09012SAndroid Build Coastguard Worker// </div> 519*d5c09012SAndroid Build Coastguard Worker// 520*d5c09012SAndroid Build Coastguard Worker// 521*d5c09012SAndroid Build Coastguard Worker// 522*d5c09012SAndroid Build Coastguard Worker// <h4>AutoML Tables</h4><div class="ui-datasection-main"><section 523*d5c09012SAndroid Build Coastguard Worker// class="selected"> 524*d5c09012SAndroid Build Coastguard Worker// 525*d5c09012SAndroid Build Coastguard Worker// See [Preparing your training 526*d5c09012SAndroid Build Coastguard Worker// data](https://cloud.google.com/automl-tables/docs/prepare) for more 527*d5c09012SAndroid Build Coastguard Worker// information. 528*d5c09012SAndroid Build Coastguard Worker// 529*d5c09012SAndroid Build Coastguard Worker// You can use either 530*d5c09012SAndroid Build Coastguard Worker// [gcs_source][google.cloud.automl.v1.InputConfig.gcs_source] or 531*d5c09012SAndroid Build Coastguard Worker// [bigquery_source][google.cloud.automl.v1.InputConfig.bigquery_source]. 532*d5c09012SAndroid Build Coastguard Worker// All input is concatenated into a 533*d5c09012SAndroid Build Coastguard Worker// single 534*d5c09012SAndroid Build Coastguard Worker// [primary_table_spec_id][google.cloud.automl.v1.TablesDatasetMetadata.primary_table_spec_id] 535*d5c09012SAndroid Build Coastguard Worker// 536*d5c09012SAndroid Build Coastguard Worker// **For gcs_source:** 537*d5c09012SAndroid Build Coastguard Worker// 538*d5c09012SAndroid Build Coastguard Worker// CSV file(s), where the first row of the first file is the header, 539*d5c09012SAndroid Build Coastguard Worker// containing unique column names. If the first row of a subsequent 540*d5c09012SAndroid Build Coastguard Worker// file is the same as the header, then it is also treated as a 541*d5c09012SAndroid Build Coastguard Worker// header. All other rows contain values for the corresponding 542*d5c09012SAndroid Build Coastguard Worker// columns. 543*d5c09012SAndroid Build Coastguard Worker// 544*d5c09012SAndroid Build Coastguard Worker// Each .CSV file by itself must be 10GB or smaller, and their total 545*d5c09012SAndroid Build Coastguard Worker// size must be 100GB or smaller. 546*d5c09012SAndroid Build Coastguard Worker// 547*d5c09012SAndroid Build Coastguard Worker// First three sample rows of a CSV file: 548*d5c09012SAndroid Build Coastguard Worker// <pre> 549*d5c09012SAndroid Build Coastguard Worker// "Id","First Name","Last Name","Dob","Addresses" 550*d5c09012SAndroid Build Coastguard Worker// "1","John","Doe","1968-01-22","[{"status":"current","address":"123_First_Avenue","city":"Seattle","state":"WA","zip":"11111","numberOfYears":"1"},{"status":"previous","address":"456_Main_Street","city":"Portland","state":"OR","zip":"22222","numberOfYears":"5"}]" 551*d5c09012SAndroid Build Coastguard Worker// "2","Jane","Doe","1980-10-16","[{"status":"current","address":"789_Any_Avenue","city":"Albany","state":"NY","zip":"33333","numberOfYears":"2"},{"status":"previous","address":"321_Main_Street","city":"Hoboken","state":"NJ","zip":"44444","numberOfYears":"3"}]} 552*d5c09012SAndroid Build Coastguard Worker// </pre> 553*d5c09012SAndroid Build Coastguard Worker// **For bigquery_source:** 554*d5c09012SAndroid Build Coastguard Worker// 555*d5c09012SAndroid Build Coastguard Worker// An URI of a BigQuery table. The user data size of the BigQuery 556*d5c09012SAndroid Build Coastguard Worker// table must be 100GB or smaller. 557*d5c09012SAndroid Build Coastguard Worker// 558*d5c09012SAndroid Build Coastguard Worker// An imported table must have between 2 and 1,000 columns, inclusive, 559*d5c09012SAndroid Build Coastguard Worker// and between 1000 and 100,000,000 rows, inclusive. There are at most 5 560*d5c09012SAndroid Build Coastguard Worker// import data running in parallel. 561*d5c09012SAndroid Build Coastguard Worker// 562*d5c09012SAndroid Build Coastguard Worker// </section> 563*d5c09012SAndroid Build Coastguard Worker// </div> 564*d5c09012SAndroid Build Coastguard Worker// 565*d5c09012SAndroid Build Coastguard Worker// 566*d5c09012SAndroid Build Coastguard Worker// **Input field definitions:** 567*d5c09012SAndroid Build Coastguard Worker// 568*d5c09012SAndroid Build Coastguard Worker// `ML_USE` 569*d5c09012SAndroid Build Coastguard Worker// : ("TRAIN" | "VALIDATE" | "TEST" | "UNASSIGNED") 570*d5c09012SAndroid Build Coastguard Worker// Describes how the given example (file) should be used for model 571*d5c09012SAndroid Build Coastguard Worker// training. "UNASSIGNED" can be used when user has no preference. 572*d5c09012SAndroid Build Coastguard Worker// 573*d5c09012SAndroid Build Coastguard Worker// `GCS_FILE_PATH` 574*d5c09012SAndroid Build Coastguard Worker// : The path to a file on Google Cloud Storage. For example, 575*d5c09012SAndroid Build Coastguard Worker// "gs://folder/image1.png". 576*d5c09012SAndroid Build Coastguard Worker// 577*d5c09012SAndroid Build Coastguard Worker// `LABEL` 578*d5c09012SAndroid Build Coastguard Worker// : A display name of an object on an image, video etc., e.g. "dog". 579*d5c09012SAndroid Build Coastguard Worker// Must be up to 32 characters long and can consist only of ASCII 580*d5c09012SAndroid Build Coastguard Worker// Latin letters A-Z and a-z, underscores(_), and ASCII digits 0-9. 581*d5c09012SAndroid Build Coastguard Worker// For each label an AnnotationSpec is created which display_name 582*d5c09012SAndroid Build Coastguard Worker// becomes the label; AnnotationSpecs are given back in predictions. 583*d5c09012SAndroid Build Coastguard Worker// 584*d5c09012SAndroid Build Coastguard Worker// `INSTANCE_ID` 585*d5c09012SAndroid Build Coastguard Worker// : A positive integer that identifies a specific instance of a 586*d5c09012SAndroid Build Coastguard Worker// labeled entity on an example. Used e.g. to track two cars on 587*d5c09012SAndroid Build Coastguard Worker// a video while being able to tell apart which one is which. 588*d5c09012SAndroid Build Coastguard Worker// 589*d5c09012SAndroid Build Coastguard Worker// `BOUNDING_BOX` 590*d5c09012SAndroid Build Coastguard Worker// : (`VERTEX,VERTEX,VERTEX,VERTEX` | `VERTEX,,,VERTEX,,`) 591*d5c09012SAndroid Build Coastguard Worker// A rectangle parallel to the frame of the example (image, 592*d5c09012SAndroid Build Coastguard Worker// video). If 4 vertices are given they are connected by edges 593*d5c09012SAndroid Build Coastguard Worker// in the order provided, if 2 are given they are recognized 594*d5c09012SAndroid Build Coastguard Worker// as diagonally opposite vertices of the rectangle. 595*d5c09012SAndroid Build Coastguard Worker// 596*d5c09012SAndroid Build Coastguard Worker// `VERTEX` 597*d5c09012SAndroid Build Coastguard Worker// : (`COORDINATE,COORDINATE`) 598*d5c09012SAndroid Build Coastguard Worker// First coordinate is horizontal (x), the second is vertical (y). 599*d5c09012SAndroid Build Coastguard Worker// 600*d5c09012SAndroid Build Coastguard Worker// `COORDINATE` 601*d5c09012SAndroid Build Coastguard Worker// : A float in 0 to 1 range, relative to total length of 602*d5c09012SAndroid Build Coastguard Worker// image or video in given dimension. For fractions the 603*d5c09012SAndroid Build Coastguard Worker// leading non-decimal 0 can be omitted (i.e. 0.3 = .3). 604*d5c09012SAndroid Build Coastguard Worker// Point 0,0 is in top left. 605*d5c09012SAndroid Build Coastguard Worker// 606*d5c09012SAndroid Build Coastguard Worker// `TIME_SEGMENT_START` 607*d5c09012SAndroid Build Coastguard Worker// : (`TIME_OFFSET`) 608*d5c09012SAndroid Build Coastguard Worker// Expresses a beginning, inclusive, of a time segment 609*d5c09012SAndroid Build Coastguard Worker// within an example that has a time dimension 610*d5c09012SAndroid Build Coastguard Worker// (e.g. video). 611*d5c09012SAndroid Build Coastguard Worker// 612*d5c09012SAndroid Build Coastguard Worker// `TIME_SEGMENT_END` 613*d5c09012SAndroid Build Coastguard Worker// : (`TIME_OFFSET`) 614*d5c09012SAndroid Build Coastguard Worker// Expresses an end, exclusive, of a time segment within 615*d5c09012SAndroid Build Coastguard Worker// n example that has a time dimension (e.g. video). 616*d5c09012SAndroid Build Coastguard Worker// 617*d5c09012SAndroid Build Coastguard Worker// `TIME_OFFSET` 618*d5c09012SAndroid Build Coastguard Worker// : A number of seconds as measured from the start of an 619*d5c09012SAndroid Build Coastguard Worker// example (e.g. video). Fractions are allowed, up to a 620*d5c09012SAndroid Build Coastguard Worker// microsecond precision. "inf" is allowed, and it means the end 621*d5c09012SAndroid Build Coastguard Worker// of the example. 622*d5c09012SAndroid Build Coastguard Worker// 623*d5c09012SAndroid Build Coastguard Worker// `TEXT_SNIPPET` 624*d5c09012SAndroid Build Coastguard Worker// : The content of a text snippet, UTF-8 encoded, enclosed within 625*d5c09012SAndroid Build Coastguard Worker// double quotes (""). 626*d5c09012SAndroid Build Coastguard Worker// 627*d5c09012SAndroid Build Coastguard Worker// `DOCUMENT` 628*d5c09012SAndroid Build Coastguard Worker// : A field that provides the textual content with document and the layout 629*d5c09012SAndroid Build Coastguard Worker// information. 630*d5c09012SAndroid Build Coastguard Worker// 631*d5c09012SAndroid Build Coastguard Worker// 632*d5c09012SAndroid Build Coastguard Worker// **Errors:** 633*d5c09012SAndroid Build Coastguard Worker// 634*d5c09012SAndroid Build Coastguard Worker// If any of the provided CSV files can't be parsed or if more than certain 635*d5c09012SAndroid Build Coastguard Worker// percent of CSV rows cannot be processed then the operation fails and 636*d5c09012SAndroid Build Coastguard Worker// nothing is imported. Regardless of overall success or failure the per-row 637*d5c09012SAndroid Build Coastguard Worker// failures, up to a certain count cap, is listed in 638*d5c09012SAndroid Build Coastguard Worker// Operation.metadata.partial_failures. 639*d5c09012SAndroid Build Coastguard Worker// 640*d5c09012SAndroid Build Coastguard Workermessage InputConfig { 641*d5c09012SAndroid Build Coastguard Worker // The source of the input. 642*d5c09012SAndroid Build Coastguard Worker oneof source { 643*d5c09012SAndroid Build Coastguard Worker // The Google Cloud Storage location for the input content. 644*d5c09012SAndroid Build Coastguard Worker // For [AutoMl.ImportData][google.cloud.automl.v1.AutoMl.ImportData], `gcs_source` points to a CSV file with 645*d5c09012SAndroid Build Coastguard Worker // a structure described in [InputConfig][google.cloud.automl.v1.InputConfig]. 646*d5c09012SAndroid Build Coastguard Worker GcsSource gcs_source = 1; 647*d5c09012SAndroid Build Coastguard Worker } 648*d5c09012SAndroid Build Coastguard Worker 649*d5c09012SAndroid Build Coastguard Worker // Additional domain-specific parameters describing the semantic of the 650*d5c09012SAndroid Build Coastguard Worker // imported data, any string must be up to 25000 651*d5c09012SAndroid Build Coastguard Worker // characters long. 652*d5c09012SAndroid Build Coastguard Worker // 653*d5c09012SAndroid Build Coastguard Worker // <h4>AutoML Tables</h4> 654*d5c09012SAndroid Build Coastguard Worker // 655*d5c09012SAndroid Build Coastguard Worker // `schema_inference_version` 656*d5c09012SAndroid Build Coastguard Worker // : (integer) This value must be supplied. 657*d5c09012SAndroid Build Coastguard Worker // The version of the 658*d5c09012SAndroid Build Coastguard Worker // algorithm to use for the initial inference of the 659*d5c09012SAndroid Build Coastguard Worker // column data types of the imported table. Allowed values: "1". 660*d5c09012SAndroid Build Coastguard Worker map<string, string> params = 2; 661*d5c09012SAndroid Build Coastguard Worker} 662*d5c09012SAndroid Build Coastguard Worker 663*d5c09012SAndroid Build Coastguard Worker// Input configuration for BatchPredict Action. 664*d5c09012SAndroid Build Coastguard Worker// 665*d5c09012SAndroid Build Coastguard Worker// The format of input depends on the ML problem of the model used for 666*d5c09012SAndroid Build Coastguard Worker// prediction. As input source the 667*d5c09012SAndroid Build Coastguard Worker// [gcs_source][google.cloud.automl.v1.InputConfig.gcs_source] 668*d5c09012SAndroid Build Coastguard Worker// is expected, unless specified otherwise. 669*d5c09012SAndroid Build Coastguard Worker// 670*d5c09012SAndroid Build Coastguard Worker// The formats are represented in EBNF with commas being literal and with 671*d5c09012SAndroid Build Coastguard Worker// non-terminal symbols defined near the end of this comment. The formats 672*d5c09012SAndroid Build Coastguard Worker// are: 673*d5c09012SAndroid Build Coastguard Worker// 674*d5c09012SAndroid Build Coastguard Worker// <h4>AutoML Vision</h4> 675*d5c09012SAndroid Build Coastguard Worker// <div class="ds-selector-tabs"><section><h5>Classification</h5> 676*d5c09012SAndroid Build Coastguard Worker// 677*d5c09012SAndroid Build Coastguard Worker// One or more CSV files where each line is a single column: 678*d5c09012SAndroid Build Coastguard Worker// 679*d5c09012SAndroid Build Coastguard Worker// GCS_FILE_PATH 680*d5c09012SAndroid Build Coastguard Worker// 681*d5c09012SAndroid Build Coastguard Worker// The Google Cloud Storage location of an image of up to 682*d5c09012SAndroid Build Coastguard Worker// 30MB in size. Supported extensions: .JPEG, .GIF, .PNG. 683*d5c09012SAndroid Build Coastguard Worker// This path is treated as the ID in the batch predict output. 684*d5c09012SAndroid Build Coastguard Worker// 685*d5c09012SAndroid Build Coastguard Worker// Sample rows: 686*d5c09012SAndroid Build Coastguard Worker// 687*d5c09012SAndroid Build Coastguard Worker// gs://folder/image1.jpeg 688*d5c09012SAndroid Build Coastguard Worker// gs://folder/image2.gif 689*d5c09012SAndroid Build Coastguard Worker// gs://folder/image3.png 690*d5c09012SAndroid Build Coastguard Worker// 691*d5c09012SAndroid Build Coastguard Worker// </section><section><h5>Object Detection</h5> 692*d5c09012SAndroid Build Coastguard Worker// 693*d5c09012SAndroid Build Coastguard Worker// One or more CSV files where each line is a single column: 694*d5c09012SAndroid Build Coastguard Worker// 695*d5c09012SAndroid Build Coastguard Worker// GCS_FILE_PATH 696*d5c09012SAndroid Build Coastguard Worker// 697*d5c09012SAndroid Build Coastguard Worker// The Google Cloud Storage location of an image of up to 698*d5c09012SAndroid Build Coastguard Worker// 30MB in size. Supported extensions: .JPEG, .GIF, .PNG. 699*d5c09012SAndroid Build Coastguard Worker// This path is treated as the ID in the batch predict output. 700*d5c09012SAndroid Build Coastguard Worker// 701*d5c09012SAndroid Build Coastguard Worker// Sample rows: 702*d5c09012SAndroid Build Coastguard Worker// 703*d5c09012SAndroid Build Coastguard Worker// gs://folder/image1.jpeg 704*d5c09012SAndroid Build Coastguard Worker// gs://folder/image2.gif 705*d5c09012SAndroid Build Coastguard Worker// gs://folder/image3.png 706*d5c09012SAndroid Build Coastguard Worker// </section> 707*d5c09012SAndroid Build Coastguard Worker// </div> 708*d5c09012SAndroid Build Coastguard Worker// 709*d5c09012SAndroid Build Coastguard Worker// <h4>AutoML Video Intelligence</h4> 710*d5c09012SAndroid Build Coastguard Worker// <div class="ds-selector-tabs"><section><h5>Classification</h5> 711*d5c09012SAndroid Build Coastguard Worker// 712*d5c09012SAndroid Build Coastguard Worker// One or more CSV files where each line is a single column: 713*d5c09012SAndroid Build Coastguard Worker// 714*d5c09012SAndroid Build Coastguard Worker// GCS_FILE_PATH,TIME_SEGMENT_START,TIME_SEGMENT_END 715*d5c09012SAndroid Build Coastguard Worker// 716*d5c09012SAndroid Build Coastguard Worker// `GCS_FILE_PATH` is the Google Cloud Storage location of video up to 50GB in 717*d5c09012SAndroid Build Coastguard Worker// size and up to 3h in duration duration. 718*d5c09012SAndroid Build Coastguard Worker// Supported extensions: .MOV, .MPEG4, .MP4, .AVI. 719*d5c09012SAndroid Build Coastguard Worker// 720*d5c09012SAndroid Build Coastguard Worker// `TIME_SEGMENT_START` and `TIME_SEGMENT_END` must be within the 721*d5c09012SAndroid Build Coastguard Worker// length of the video, and the end time must be after the start time. 722*d5c09012SAndroid Build Coastguard Worker// 723*d5c09012SAndroid Build Coastguard Worker// Sample rows: 724*d5c09012SAndroid Build Coastguard Worker// 725*d5c09012SAndroid Build Coastguard Worker// gs://folder/video1.mp4,10,40 726*d5c09012SAndroid Build Coastguard Worker// gs://folder/video1.mp4,20,60 727*d5c09012SAndroid Build Coastguard Worker// gs://folder/vid2.mov,0,inf 728*d5c09012SAndroid Build Coastguard Worker// 729*d5c09012SAndroid Build Coastguard Worker// </section><section><h5>Object Tracking</h5> 730*d5c09012SAndroid Build Coastguard Worker// 731*d5c09012SAndroid Build Coastguard Worker// One or more CSV files where each line is a single column: 732*d5c09012SAndroid Build Coastguard Worker// 733*d5c09012SAndroid Build Coastguard Worker// GCS_FILE_PATH,TIME_SEGMENT_START,TIME_SEGMENT_END 734*d5c09012SAndroid Build Coastguard Worker// 735*d5c09012SAndroid Build Coastguard Worker// `GCS_FILE_PATH` is the Google Cloud Storage location of video up to 50GB in 736*d5c09012SAndroid Build Coastguard Worker// size and up to 3h in duration duration. 737*d5c09012SAndroid Build Coastguard Worker// Supported extensions: .MOV, .MPEG4, .MP4, .AVI. 738*d5c09012SAndroid Build Coastguard Worker// 739*d5c09012SAndroid Build Coastguard Worker// `TIME_SEGMENT_START` and `TIME_SEGMENT_END` must be within the 740*d5c09012SAndroid Build Coastguard Worker// length of the video, and the end time must be after the start time. 741*d5c09012SAndroid Build Coastguard Worker// 742*d5c09012SAndroid Build Coastguard Worker// Sample rows: 743*d5c09012SAndroid Build Coastguard Worker// 744*d5c09012SAndroid Build Coastguard Worker// gs://folder/video1.mp4,10,40 745*d5c09012SAndroid Build Coastguard Worker// gs://folder/video1.mp4,20,60 746*d5c09012SAndroid Build Coastguard Worker// gs://folder/vid2.mov,0,inf 747*d5c09012SAndroid Build Coastguard Worker// </section> 748*d5c09012SAndroid Build Coastguard Worker// </div> 749*d5c09012SAndroid Build Coastguard Worker// 750*d5c09012SAndroid Build Coastguard Worker// <h4>AutoML Natural Language</h4> 751*d5c09012SAndroid Build Coastguard Worker// <div class="ds-selector-tabs"><section><h5>Classification</h5> 752*d5c09012SAndroid Build Coastguard Worker// 753*d5c09012SAndroid Build Coastguard Worker// One or more CSV files where each line is a single column: 754*d5c09012SAndroid Build Coastguard Worker// 755*d5c09012SAndroid Build Coastguard Worker// GCS_FILE_PATH 756*d5c09012SAndroid Build Coastguard Worker// 757*d5c09012SAndroid Build Coastguard Worker// `GCS_FILE_PATH` is the Google Cloud Storage location of a text file. 758*d5c09012SAndroid Build Coastguard Worker// Supported file extensions: .TXT, .PDF, .TIF, .TIFF 759*d5c09012SAndroid Build Coastguard Worker// 760*d5c09012SAndroid Build Coastguard Worker// Text files can be no larger than 10MB in size. 761*d5c09012SAndroid Build Coastguard Worker// 762*d5c09012SAndroid Build Coastguard Worker// Sample rows: 763*d5c09012SAndroid Build Coastguard Worker// 764*d5c09012SAndroid Build Coastguard Worker// gs://folder/text1.txt 765*d5c09012SAndroid Build Coastguard Worker// gs://folder/text2.pdf 766*d5c09012SAndroid Build Coastguard Worker// gs://folder/text3.tif 767*d5c09012SAndroid Build Coastguard Worker// 768*d5c09012SAndroid Build Coastguard Worker// </section><section><h5>Sentiment Analysis</h5> 769*d5c09012SAndroid Build Coastguard Worker// One or more CSV files where each line is a single column: 770*d5c09012SAndroid Build Coastguard Worker// 771*d5c09012SAndroid Build Coastguard Worker// GCS_FILE_PATH 772*d5c09012SAndroid Build Coastguard Worker// 773*d5c09012SAndroid Build Coastguard Worker// `GCS_FILE_PATH` is the Google Cloud Storage location of a text file. 774*d5c09012SAndroid Build Coastguard Worker// Supported file extensions: .TXT, .PDF, .TIF, .TIFF 775*d5c09012SAndroid Build Coastguard Worker// 776*d5c09012SAndroid Build Coastguard Worker// Text files can be no larger than 128kB in size. 777*d5c09012SAndroid Build Coastguard Worker// 778*d5c09012SAndroid Build Coastguard Worker// Sample rows: 779*d5c09012SAndroid Build Coastguard Worker// 780*d5c09012SAndroid Build Coastguard Worker// gs://folder/text1.txt 781*d5c09012SAndroid Build Coastguard Worker// gs://folder/text2.pdf 782*d5c09012SAndroid Build Coastguard Worker// gs://folder/text3.tif 783*d5c09012SAndroid Build Coastguard Worker// 784*d5c09012SAndroid Build Coastguard Worker// </section><section><h5>Entity Extraction</h5> 785*d5c09012SAndroid Build Coastguard Worker// 786*d5c09012SAndroid Build Coastguard Worker// One or more JSONL (JSON Lines) files that either provide inline text or 787*d5c09012SAndroid Build Coastguard Worker// documents. You can only use one format, either inline text or documents, 788*d5c09012SAndroid Build Coastguard Worker// for a single call to [AutoMl.BatchPredict]. 789*d5c09012SAndroid Build Coastguard Worker// 790*d5c09012SAndroid Build Coastguard Worker// Each JSONL file contains a per line a proto that 791*d5c09012SAndroid Build Coastguard Worker// wraps a temporary user-assigned TextSnippet ID (string up to 2000 792*d5c09012SAndroid Build Coastguard Worker// characters long) called "id", a TextSnippet proto (in 793*d5c09012SAndroid Build Coastguard Worker// JSON representation) and zero or more TextFeature protos. Any given 794*d5c09012SAndroid Build Coastguard Worker// text snippet content must have 30,000 characters or less, and also 795*d5c09012SAndroid Build Coastguard Worker// be UTF-8 NFC encoded (ASCII already is). The IDs provided should be 796*d5c09012SAndroid Build Coastguard Worker// unique. 797*d5c09012SAndroid Build Coastguard Worker// 798*d5c09012SAndroid Build Coastguard Worker// Each document JSONL file contains, per line, a proto that wraps a Document 799*d5c09012SAndroid Build Coastguard Worker// proto with `input_config` set. Each document cannot exceed 2MB in size. 800*d5c09012SAndroid Build Coastguard Worker// 801*d5c09012SAndroid Build Coastguard Worker// Supported document extensions: .PDF, .TIF, .TIFF 802*d5c09012SAndroid Build Coastguard Worker// 803*d5c09012SAndroid Build Coastguard Worker// Each JSONL file must not exceed 100MB in size, and no more than 20 804*d5c09012SAndroid Build Coastguard Worker// JSONL files may be passed. 805*d5c09012SAndroid Build Coastguard Worker// 806*d5c09012SAndroid Build Coastguard Worker// Sample inline JSONL file (Shown with artificial line 807*d5c09012SAndroid Build Coastguard Worker// breaks. Actual line breaks are denoted by "\n".): 808*d5c09012SAndroid Build Coastguard Worker// 809*d5c09012SAndroid Build Coastguard Worker// { 810*d5c09012SAndroid Build Coastguard Worker// "id": "my_first_id", 811*d5c09012SAndroid Build Coastguard Worker// "text_snippet": { "content": "dog car cat"}, 812*d5c09012SAndroid Build Coastguard Worker// "text_features": [ 813*d5c09012SAndroid Build Coastguard Worker// { 814*d5c09012SAndroid Build Coastguard Worker// "text_segment": {"start_offset": 4, "end_offset": 6}, 815*d5c09012SAndroid Build Coastguard Worker// "structural_type": PARAGRAPH, 816*d5c09012SAndroid Build Coastguard Worker// "bounding_poly": { 817*d5c09012SAndroid Build Coastguard Worker// "normalized_vertices": [ 818*d5c09012SAndroid Build Coastguard Worker// {"x": 0.1, "y": 0.1}, 819*d5c09012SAndroid Build Coastguard Worker// {"x": 0.1, "y": 0.3}, 820*d5c09012SAndroid Build Coastguard Worker// {"x": 0.3, "y": 0.3}, 821*d5c09012SAndroid Build Coastguard Worker// {"x": 0.3, "y": 0.1}, 822*d5c09012SAndroid Build Coastguard Worker// ] 823*d5c09012SAndroid Build Coastguard Worker// }, 824*d5c09012SAndroid Build Coastguard Worker// } 825*d5c09012SAndroid Build Coastguard Worker// ], 826*d5c09012SAndroid Build Coastguard Worker// }\n 827*d5c09012SAndroid Build Coastguard Worker// { 828*d5c09012SAndroid Build Coastguard Worker// "id": "2", 829*d5c09012SAndroid Build Coastguard Worker// "text_snippet": { 830*d5c09012SAndroid Build Coastguard Worker// "content": "Extended sample content", 831*d5c09012SAndroid Build Coastguard Worker// "mime_type": "text/plain" 832*d5c09012SAndroid Build Coastguard Worker// } 833*d5c09012SAndroid Build Coastguard Worker// } 834*d5c09012SAndroid Build Coastguard Worker// 835*d5c09012SAndroid Build Coastguard Worker// Sample document JSONL file (Shown with artificial line 836*d5c09012SAndroid Build Coastguard Worker// breaks. Actual line breaks are denoted by "\n".): 837*d5c09012SAndroid Build Coastguard Worker// 838*d5c09012SAndroid Build Coastguard Worker// { 839*d5c09012SAndroid Build Coastguard Worker// "document": { 840*d5c09012SAndroid Build Coastguard Worker// "input_config": { 841*d5c09012SAndroid Build Coastguard Worker// "gcs_source": { "input_uris": [ "gs://folder/document1.pdf" ] 842*d5c09012SAndroid Build Coastguard Worker// } 843*d5c09012SAndroid Build Coastguard Worker// } 844*d5c09012SAndroid Build Coastguard Worker// } 845*d5c09012SAndroid Build Coastguard Worker// }\n 846*d5c09012SAndroid Build Coastguard Worker// { 847*d5c09012SAndroid Build Coastguard Worker// "document": { 848*d5c09012SAndroid Build Coastguard Worker// "input_config": { 849*d5c09012SAndroid Build Coastguard Worker// "gcs_source": { "input_uris": [ "gs://folder/document2.tif" ] 850*d5c09012SAndroid Build Coastguard Worker// } 851*d5c09012SAndroid Build Coastguard Worker// } 852*d5c09012SAndroid Build Coastguard Worker// } 853*d5c09012SAndroid Build Coastguard Worker// } 854*d5c09012SAndroid Build Coastguard Worker// </section> 855*d5c09012SAndroid Build Coastguard Worker// </div> 856*d5c09012SAndroid Build Coastguard Worker// 857*d5c09012SAndroid Build Coastguard Worker// <h4>AutoML Tables</h4><div class="ui-datasection-main"><section 858*d5c09012SAndroid Build Coastguard Worker// class="selected"> 859*d5c09012SAndroid Build Coastguard Worker// 860*d5c09012SAndroid Build Coastguard Worker// See [Preparing your training 861*d5c09012SAndroid Build Coastguard Worker// data](https://cloud.google.com/automl-tables/docs/predict-batch) for more 862*d5c09012SAndroid Build Coastguard Worker// information. 863*d5c09012SAndroid Build Coastguard Worker// 864*d5c09012SAndroid Build Coastguard Worker// You can use either 865*d5c09012SAndroid Build Coastguard Worker// [gcs_source][google.cloud.automl.v1.BatchPredictInputConfig.gcs_source] 866*d5c09012SAndroid Build Coastguard Worker// or 867*d5c09012SAndroid Build Coastguard Worker// [bigquery_source][BatchPredictInputConfig.bigquery_source]. 868*d5c09012SAndroid Build Coastguard Worker// 869*d5c09012SAndroid Build Coastguard Worker// **For gcs_source:** 870*d5c09012SAndroid Build Coastguard Worker// 871*d5c09012SAndroid Build Coastguard Worker// CSV file(s), each by itself 10GB or smaller and total size must be 872*d5c09012SAndroid Build Coastguard Worker// 100GB or smaller, where first file must have a header containing 873*d5c09012SAndroid Build Coastguard Worker// column names. If the first row of a subsequent file is the same as 874*d5c09012SAndroid Build Coastguard Worker// the header, then it is also treated as a header. All other rows 875*d5c09012SAndroid Build Coastguard Worker// contain values for the corresponding columns. 876*d5c09012SAndroid Build Coastguard Worker// 877*d5c09012SAndroid Build Coastguard Worker// The column names must contain the model's 878*d5c09012SAndroid Build Coastguard Worker// [input_feature_column_specs'][google.cloud.automl.v1.TablesModelMetadata.input_feature_column_specs] 879*d5c09012SAndroid Build Coastguard Worker// [display_name-s][google.cloud.automl.v1.ColumnSpec.display_name] 880*d5c09012SAndroid Build Coastguard Worker// (order doesn't matter). The columns corresponding to the model's 881*d5c09012SAndroid Build Coastguard Worker// input feature column specs must contain values compatible with the 882*d5c09012SAndroid Build Coastguard Worker// column spec's data types. Prediction on all the rows, i.e. the CSV 883*d5c09012SAndroid Build Coastguard Worker// lines, will be attempted. 884*d5c09012SAndroid Build Coastguard Worker// 885*d5c09012SAndroid Build Coastguard Worker// 886*d5c09012SAndroid Build Coastguard Worker// Sample rows from a CSV file: 887*d5c09012SAndroid Build Coastguard Worker// <pre> 888*d5c09012SAndroid Build Coastguard Worker// "First Name","Last Name","Dob","Addresses" 889*d5c09012SAndroid Build Coastguard Worker// "John","Doe","1968-01-22","[{"status":"current","address":"123_First_Avenue","city":"Seattle","state":"WA","zip":"11111","numberOfYears":"1"},{"status":"previous","address":"456_Main_Street","city":"Portland","state":"OR","zip":"22222","numberOfYears":"5"}]" 890*d5c09012SAndroid Build Coastguard Worker// "Jane","Doe","1980-10-16","[{"status":"current","address":"789_Any_Avenue","city":"Albany","state":"NY","zip":"33333","numberOfYears":"2"},{"status":"previous","address":"321_Main_Street","city":"Hoboken","state":"NJ","zip":"44444","numberOfYears":"3"}]} 891*d5c09012SAndroid Build Coastguard Worker// </pre> 892*d5c09012SAndroid Build Coastguard Worker// **For bigquery_source:** 893*d5c09012SAndroid Build Coastguard Worker// 894*d5c09012SAndroid Build Coastguard Worker// The URI of a BigQuery table. The user data size of the BigQuery 895*d5c09012SAndroid Build Coastguard Worker// table must be 100GB or smaller. 896*d5c09012SAndroid Build Coastguard Worker// 897*d5c09012SAndroid Build Coastguard Worker// The column names must contain the model's 898*d5c09012SAndroid Build Coastguard Worker// [input_feature_column_specs'][google.cloud.automl.v1.TablesModelMetadata.input_feature_column_specs] 899*d5c09012SAndroid Build Coastguard Worker// [display_name-s][google.cloud.automl.v1.ColumnSpec.display_name] 900*d5c09012SAndroid Build Coastguard Worker// (order doesn't matter). The columns corresponding to the model's 901*d5c09012SAndroid Build Coastguard Worker// input feature column specs must contain values compatible with the 902*d5c09012SAndroid Build Coastguard Worker// column spec's data types. Prediction on all the rows of the table 903*d5c09012SAndroid Build Coastguard Worker// will be attempted. 904*d5c09012SAndroid Build Coastguard Worker// </section> 905*d5c09012SAndroid Build Coastguard Worker// </div> 906*d5c09012SAndroid Build Coastguard Worker// 907*d5c09012SAndroid Build Coastguard Worker// **Input field definitions:** 908*d5c09012SAndroid Build Coastguard Worker// 909*d5c09012SAndroid Build Coastguard Worker// `GCS_FILE_PATH` 910*d5c09012SAndroid Build Coastguard Worker// : The path to a file on Google Cloud Storage. For example, 911*d5c09012SAndroid Build Coastguard Worker// "gs://folder/video.avi". 912*d5c09012SAndroid Build Coastguard Worker// 913*d5c09012SAndroid Build Coastguard Worker// `TIME_SEGMENT_START` 914*d5c09012SAndroid Build Coastguard Worker// : (`TIME_OFFSET`) 915*d5c09012SAndroid Build Coastguard Worker// Expresses a beginning, inclusive, of a time segment 916*d5c09012SAndroid Build Coastguard Worker// within an example that has a time dimension 917*d5c09012SAndroid Build Coastguard Worker// (e.g. video). 918*d5c09012SAndroid Build Coastguard Worker// 919*d5c09012SAndroid Build Coastguard Worker// `TIME_SEGMENT_END` 920*d5c09012SAndroid Build Coastguard Worker// : (`TIME_OFFSET`) 921*d5c09012SAndroid Build Coastguard Worker// Expresses an end, exclusive, of a time segment within 922*d5c09012SAndroid Build Coastguard Worker// n example that has a time dimension (e.g. video). 923*d5c09012SAndroid Build Coastguard Worker// 924*d5c09012SAndroid Build Coastguard Worker// `TIME_OFFSET` 925*d5c09012SAndroid Build Coastguard Worker// : A number of seconds as measured from the start of an 926*d5c09012SAndroid Build Coastguard Worker// example (e.g. video). Fractions are allowed, up to a 927*d5c09012SAndroid Build Coastguard Worker// microsecond precision. "inf" is allowed, and it means the end 928*d5c09012SAndroid Build Coastguard Worker// of the example. 929*d5c09012SAndroid Build Coastguard Worker// 930*d5c09012SAndroid Build Coastguard Worker// **Errors:** 931*d5c09012SAndroid Build Coastguard Worker// 932*d5c09012SAndroid Build Coastguard Worker// If any of the provided CSV files can't be parsed or if more than certain 933*d5c09012SAndroid Build Coastguard Worker// percent of CSV rows cannot be processed then the operation fails and 934*d5c09012SAndroid Build Coastguard Worker// prediction does not happen. Regardless of overall success or failure the 935*d5c09012SAndroid Build Coastguard Worker// per-row failures, up to a certain count cap, will be listed in 936*d5c09012SAndroid Build Coastguard Worker// Operation.metadata.partial_failures. 937*d5c09012SAndroid Build Coastguard Workermessage BatchPredictInputConfig { 938*d5c09012SAndroid Build Coastguard Worker // The source of the input. 939*d5c09012SAndroid Build Coastguard Worker oneof source { 940*d5c09012SAndroid Build Coastguard Worker // Required. The Google Cloud Storage location for the input content. 941*d5c09012SAndroid Build Coastguard Worker GcsSource gcs_source = 1 [(google.api.field_behavior) = REQUIRED]; 942*d5c09012SAndroid Build Coastguard Worker } 943*d5c09012SAndroid Build Coastguard Worker} 944*d5c09012SAndroid Build Coastguard Worker 945*d5c09012SAndroid Build Coastguard Worker// Input configuration of a [Document][google.cloud.automl.v1.Document]. 946*d5c09012SAndroid Build Coastguard Workermessage DocumentInputConfig { 947*d5c09012SAndroid Build Coastguard Worker // The Google Cloud Storage location of the document file. Only a single path 948*d5c09012SAndroid Build Coastguard Worker // should be given. 949*d5c09012SAndroid Build Coastguard Worker // 950*d5c09012SAndroid Build Coastguard Worker // Max supported size: 512MB. 951*d5c09012SAndroid Build Coastguard Worker // 952*d5c09012SAndroid Build Coastguard Worker // Supported extensions: .PDF. 953*d5c09012SAndroid Build Coastguard Worker GcsSource gcs_source = 1; 954*d5c09012SAndroid Build Coastguard Worker} 955*d5c09012SAndroid Build Coastguard Worker 956*d5c09012SAndroid Build Coastguard Worker// * For Translation: 957*d5c09012SAndroid Build Coastguard Worker// CSV file `translation.csv`, with each line in format: 958*d5c09012SAndroid Build Coastguard Worker// ML_USE,GCS_FILE_PATH 959*d5c09012SAndroid Build Coastguard Worker// GCS_FILE_PATH leads to a .TSV file which describes examples that have 960*d5c09012SAndroid Build Coastguard Worker// given ML_USE, using the following row format per line: 961*d5c09012SAndroid Build Coastguard Worker// TEXT_SNIPPET (in source language) \t TEXT_SNIPPET (in target 962*d5c09012SAndroid Build Coastguard Worker// language) 963*d5c09012SAndroid Build Coastguard Worker// 964*d5c09012SAndroid Build Coastguard Worker// * For Tables: 965*d5c09012SAndroid Build Coastguard Worker// Output depends on whether the dataset was imported from Google Cloud 966*d5c09012SAndroid Build Coastguard Worker// Storage or BigQuery. 967*d5c09012SAndroid Build Coastguard Worker// Google Cloud Storage case: 968*d5c09012SAndroid Build Coastguard Worker// [gcs_destination][google.cloud.automl.v1p1beta.OutputConfig.gcs_destination] 969*d5c09012SAndroid Build Coastguard Worker// must be set. Exported are CSV file(s) `tables_1.csv`, 970*d5c09012SAndroid Build Coastguard Worker// `tables_2.csv`,...,`tables_N.csv` with each having as header line 971*d5c09012SAndroid Build Coastguard Worker// the table's column names, and all other lines contain values for 972*d5c09012SAndroid Build Coastguard Worker// the header columns. 973*d5c09012SAndroid Build Coastguard Worker// BigQuery case: 974*d5c09012SAndroid Build Coastguard Worker// [bigquery_destination][google.cloud.automl.v1p1beta.OutputConfig.bigquery_destination] 975*d5c09012SAndroid Build Coastguard Worker// pointing to a BigQuery project must be set. In the given project a 976*d5c09012SAndroid Build Coastguard Worker// new dataset will be created with name 977*d5c09012SAndroid Build Coastguard Worker// `export_data_<automl-dataset-display-name>_<timestamp-of-export-call>` 978*d5c09012SAndroid Build Coastguard Worker// where <automl-dataset-display-name> will be made 979*d5c09012SAndroid Build Coastguard Worker// BigQuery-dataset-name compatible (e.g. most special characters will 980*d5c09012SAndroid Build Coastguard Worker// become underscores), and timestamp will be in 981*d5c09012SAndroid Build Coastguard Worker// YYYY_MM_DDThh_mm_ss_sssZ "based on ISO-8601" format. In that 982*d5c09012SAndroid Build Coastguard Worker// dataset a new table called `primary_table` will be created, and 983*d5c09012SAndroid Build Coastguard Worker// filled with precisely the same data as this obtained on import. 984*d5c09012SAndroid Build Coastguard Workermessage OutputConfig { 985*d5c09012SAndroid Build Coastguard Worker // The destination of the output. 986*d5c09012SAndroid Build Coastguard Worker oneof destination { 987*d5c09012SAndroid Build Coastguard Worker // Required. The Google Cloud Storage location where the output is to be written to. 988*d5c09012SAndroid Build Coastguard Worker // For Image Object Detection, Text Extraction, Video Classification and 989*d5c09012SAndroid Build Coastguard Worker // Tables, in the given directory a new directory will be created with name: 990*d5c09012SAndroid Build Coastguard Worker // export_data-<dataset-display-name>-<timestamp-of-export-call> where 991*d5c09012SAndroid Build Coastguard Worker // timestamp is in YYYY-MM-DDThh:mm:ss.sssZ ISO-8601 format. All export 992*d5c09012SAndroid Build Coastguard Worker // output will be written into that directory. 993*d5c09012SAndroid Build Coastguard Worker GcsDestination gcs_destination = 1 [(google.api.field_behavior) = REQUIRED]; 994*d5c09012SAndroid Build Coastguard Worker } 995*d5c09012SAndroid Build Coastguard Worker} 996*d5c09012SAndroid Build Coastguard Worker 997*d5c09012SAndroid Build Coastguard Worker// Output configuration for BatchPredict Action. 998*d5c09012SAndroid Build Coastguard Worker// 999*d5c09012SAndroid Build Coastguard Worker// As destination the 1000*d5c09012SAndroid Build Coastguard Worker// [gcs_destination][google.cloud.automl.v1.BatchPredictOutputConfig.gcs_destination] 1001*d5c09012SAndroid Build Coastguard Worker// must be set unless specified otherwise for a domain. If gcs_destination is 1002*d5c09012SAndroid Build Coastguard Worker// set then in the given directory a new directory is created. Its name 1003*d5c09012SAndroid Build Coastguard Worker// will be 1004*d5c09012SAndroid Build Coastguard Worker// "prediction-<model-display-name>-<timestamp-of-prediction-call>", 1005*d5c09012SAndroid Build Coastguard Worker// where timestamp is in YYYY-MM-DDThh:mm:ss.sssZ ISO-8601 format. The contents 1006*d5c09012SAndroid Build Coastguard Worker// of it depends on the ML problem the predictions are made for. 1007*d5c09012SAndroid Build Coastguard Worker// 1008*d5c09012SAndroid Build Coastguard Worker// * For Image Classification: 1009*d5c09012SAndroid Build Coastguard Worker// In the created directory files `image_classification_1.jsonl`, 1010*d5c09012SAndroid Build Coastguard Worker// `image_classification_2.jsonl`,...,`image_classification_N.jsonl` 1011*d5c09012SAndroid Build Coastguard Worker// will be created, where N may be 1, and depends on the 1012*d5c09012SAndroid Build Coastguard Worker// total number of the successfully predicted images and annotations. 1013*d5c09012SAndroid Build Coastguard Worker// A single image will be listed only once with all its annotations, 1014*d5c09012SAndroid Build Coastguard Worker// and its annotations will never be split across files. 1015*d5c09012SAndroid Build Coastguard Worker// Each .JSONL file will contain, per line, a JSON representation of a 1016*d5c09012SAndroid Build Coastguard Worker// proto that wraps image's "ID" : "<id_value>" followed by a list of 1017*d5c09012SAndroid Build Coastguard Worker// zero or more AnnotationPayload protos (called annotations), which 1018*d5c09012SAndroid Build Coastguard Worker// have classification detail populated. 1019*d5c09012SAndroid Build Coastguard Worker// If prediction for any image failed (partially or completely), then an 1020*d5c09012SAndroid Build Coastguard Worker// additional `errors_1.jsonl`, `errors_2.jsonl`,..., `errors_N.jsonl` 1021*d5c09012SAndroid Build Coastguard Worker// files will be created (N depends on total number of failed 1022*d5c09012SAndroid Build Coastguard Worker// predictions). These files will have a JSON representation of a proto 1023*d5c09012SAndroid Build Coastguard Worker// that wraps the same "ID" : "<id_value>" but here followed by 1024*d5c09012SAndroid Build Coastguard Worker// exactly one 1025*d5c09012SAndroid Build Coastguard Worker// [`google.rpc.Status`](https://github.com/googleapis/googleapis/blob/master/google/rpc/status.proto) 1026*d5c09012SAndroid Build Coastguard Worker// containing only `code` and `message`fields. 1027*d5c09012SAndroid Build Coastguard Worker// 1028*d5c09012SAndroid Build Coastguard Worker// * For Image Object Detection: 1029*d5c09012SAndroid Build Coastguard Worker// In the created directory files `image_object_detection_1.jsonl`, 1030*d5c09012SAndroid Build Coastguard Worker// `image_object_detection_2.jsonl`,...,`image_object_detection_N.jsonl` 1031*d5c09012SAndroid Build Coastguard Worker// will be created, where N may be 1, and depends on the 1032*d5c09012SAndroid Build Coastguard Worker// total number of the successfully predicted images and annotations. 1033*d5c09012SAndroid Build Coastguard Worker// Each .JSONL file will contain, per line, a JSON representation of a 1034*d5c09012SAndroid Build Coastguard Worker// proto that wraps image's "ID" : "<id_value>" followed by a list of 1035*d5c09012SAndroid Build Coastguard Worker// zero or more AnnotationPayload protos (called annotations), which 1036*d5c09012SAndroid Build Coastguard Worker// have image_object_detection detail populated. A single image will 1037*d5c09012SAndroid Build Coastguard Worker// be listed only once with all its annotations, and its annotations 1038*d5c09012SAndroid Build Coastguard Worker// will never be split across files. 1039*d5c09012SAndroid Build Coastguard Worker// If prediction for any image failed (partially or completely), then 1040*d5c09012SAndroid Build Coastguard Worker// additional `errors_1.jsonl`, `errors_2.jsonl`,..., `errors_N.jsonl` 1041*d5c09012SAndroid Build Coastguard Worker// files will be created (N depends on total number of failed 1042*d5c09012SAndroid Build Coastguard Worker// predictions). These files will have a JSON representation of a proto 1043*d5c09012SAndroid Build Coastguard Worker// that wraps the same "ID" : "<id_value>" but here followed by 1044*d5c09012SAndroid Build Coastguard Worker// exactly one 1045*d5c09012SAndroid Build Coastguard Worker// [`google.rpc.Status`](https://github.com/googleapis/googleapis/blob/master/google/rpc/status.proto) 1046*d5c09012SAndroid Build Coastguard Worker// containing only `code` and `message`fields. 1047*d5c09012SAndroid Build Coastguard Worker// * For Video Classification: 1048*d5c09012SAndroid Build Coastguard Worker// In the created directory a video_classification.csv file, and a .JSON 1049*d5c09012SAndroid Build Coastguard Worker// file per each video classification requested in the input (i.e. each 1050*d5c09012SAndroid Build Coastguard Worker// line in given CSV(s)), will be created. 1051*d5c09012SAndroid Build Coastguard Worker// 1052*d5c09012SAndroid Build Coastguard Worker// The format of video_classification.csv is: 1053*d5c09012SAndroid Build Coastguard Worker// GCS_FILE_PATH,TIME_SEGMENT_START,TIME_SEGMENT_END,JSON_FILE_NAME,STATUS 1054*d5c09012SAndroid Build Coastguard Worker// where: 1055*d5c09012SAndroid Build Coastguard Worker// GCS_FILE_PATH,TIME_SEGMENT_START,TIME_SEGMENT_END = matches 1 to 1 1056*d5c09012SAndroid Build Coastguard Worker// the prediction input lines (i.e. video_classification.csv has 1057*d5c09012SAndroid Build Coastguard Worker// precisely the same number of lines as the prediction input had.) 1058*d5c09012SAndroid Build Coastguard Worker// JSON_FILE_NAME = Name of .JSON file in the output directory, which 1059*d5c09012SAndroid Build Coastguard Worker// contains prediction responses for the video time segment. 1060*d5c09012SAndroid Build Coastguard Worker// STATUS = "OK" if prediction completed successfully, or an error code 1061*d5c09012SAndroid Build Coastguard Worker// with message otherwise. If STATUS is not "OK" then the .JSON file 1062*d5c09012SAndroid Build Coastguard Worker// for that line may not exist or be empty. 1063*d5c09012SAndroid Build Coastguard Worker// 1064*d5c09012SAndroid Build Coastguard Worker// Each .JSON file, assuming STATUS is "OK", will contain a list of 1065*d5c09012SAndroid Build Coastguard Worker// AnnotationPayload protos in JSON format, which are the predictions 1066*d5c09012SAndroid Build Coastguard Worker// for the video time segment the file is assigned to in the 1067*d5c09012SAndroid Build Coastguard Worker// video_classification.csv. All AnnotationPayload protos will have 1068*d5c09012SAndroid Build Coastguard Worker// video_classification field set, and will be sorted by 1069*d5c09012SAndroid Build Coastguard Worker// video_classification.type field (note that the returned types are 1070*d5c09012SAndroid Build Coastguard Worker// governed by `classifaction_types` parameter in 1071*d5c09012SAndroid Build Coastguard Worker// [PredictService.BatchPredictRequest.params][]). 1072*d5c09012SAndroid Build Coastguard Worker// 1073*d5c09012SAndroid Build Coastguard Worker// * For Video Object Tracking: 1074*d5c09012SAndroid Build Coastguard Worker// In the created directory a video_object_tracking.csv file will be 1075*d5c09012SAndroid Build Coastguard Worker// created, and multiple files video_object_trackinng_1.json, 1076*d5c09012SAndroid Build Coastguard Worker// video_object_trackinng_2.json,..., video_object_trackinng_N.json, 1077*d5c09012SAndroid Build Coastguard Worker// where N is the number of requests in the input (i.e. the number of 1078*d5c09012SAndroid Build Coastguard Worker// lines in given CSV(s)). 1079*d5c09012SAndroid Build Coastguard Worker// 1080*d5c09012SAndroid Build Coastguard Worker// The format of video_object_tracking.csv is: 1081*d5c09012SAndroid Build Coastguard Worker// GCS_FILE_PATH,TIME_SEGMENT_START,TIME_SEGMENT_END,JSON_FILE_NAME,STATUS 1082*d5c09012SAndroid Build Coastguard Worker// where: 1083*d5c09012SAndroid Build Coastguard Worker// GCS_FILE_PATH,TIME_SEGMENT_START,TIME_SEGMENT_END = matches 1 to 1 1084*d5c09012SAndroid Build Coastguard Worker// the prediction input lines (i.e. video_object_tracking.csv has 1085*d5c09012SAndroid Build Coastguard Worker// precisely the same number of lines as the prediction input had.) 1086*d5c09012SAndroid Build Coastguard Worker// JSON_FILE_NAME = Name of .JSON file in the output directory, which 1087*d5c09012SAndroid Build Coastguard Worker// contains prediction responses for the video time segment. 1088*d5c09012SAndroid Build Coastguard Worker// STATUS = "OK" if prediction completed successfully, or an error 1089*d5c09012SAndroid Build Coastguard Worker// code with message otherwise. If STATUS is not "OK" then the .JSON 1090*d5c09012SAndroid Build Coastguard Worker// file for that line may not exist or be empty. 1091*d5c09012SAndroid Build Coastguard Worker// 1092*d5c09012SAndroid Build Coastguard Worker// Each .JSON file, assuming STATUS is "OK", will contain a list of 1093*d5c09012SAndroid Build Coastguard Worker// AnnotationPayload protos in JSON format, which are the predictions 1094*d5c09012SAndroid Build Coastguard Worker// for each frame of the video time segment the file is assigned to in 1095*d5c09012SAndroid Build Coastguard Worker// video_object_tracking.csv. All AnnotationPayload protos will have 1096*d5c09012SAndroid Build Coastguard Worker// video_object_tracking field set. 1097*d5c09012SAndroid Build Coastguard Worker// * For Text Classification: 1098*d5c09012SAndroid Build Coastguard Worker// In the created directory files `text_classification_1.jsonl`, 1099*d5c09012SAndroid Build Coastguard Worker// `text_classification_2.jsonl`,...,`text_classification_N.jsonl` 1100*d5c09012SAndroid Build Coastguard Worker// will be created, where N may be 1, and depends on the 1101*d5c09012SAndroid Build Coastguard Worker// total number of inputs and annotations found. 1102*d5c09012SAndroid Build Coastguard Worker// 1103*d5c09012SAndroid Build Coastguard Worker// Each .JSONL file will contain, per line, a JSON representation of a 1104*d5c09012SAndroid Build Coastguard Worker// proto that wraps input text file (or document) in 1105*d5c09012SAndroid Build Coastguard Worker// the text snippet (or document) proto and a list of 1106*d5c09012SAndroid Build Coastguard Worker// zero or more AnnotationPayload protos (called annotations), which 1107*d5c09012SAndroid Build Coastguard Worker// have classification detail populated. A single text file (or 1108*d5c09012SAndroid Build Coastguard Worker// document) will be listed only once with all its annotations, and its 1109*d5c09012SAndroid Build Coastguard Worker// annotations will never be split across files. 1110*d5c09012SAndroid Build Coastguard Worker// 1111*d5c09012SAndroid Build Coastguard Worker// If prediction for any input file (or document) failed (partially or 1112*d5c09012SAndroid Build Coastguard Worker// completely), then additional `errors_1.jsonl`, `errors_2.jsonl`,..., 1113*d5c09012SAndroid Build Coastguard Worker// `errors_N.jsonl` files will be created (N depends on total number of 1114*d5c09012SAndroid Build Coastguard Worker// failed predictions). These files will have a JSON representation of a 1115*d5c09012SAndroid Build Coastguard Worker// proto that wraps input file followed by exactly one 1116*d5c09012SAndroid Build Coastguard Worker// [`google.rpc.Status`](https://github.com/googleapis/googleapis/blob/master/google/rpc/status.proto) 1117*d5c09012SAndroid Build Coastguard Worker// containing only `code` and `message`. 1118*d5c09012SAndroid Build Coastguard Worker// 1119*d5c09012SAndroid Build Coastguard Worker// * For Text Sentiment: 1120*d5c09012SAndroid Build Coastguard Worker// In the created directory files `text_sentiment_1.jsonl`, 1121*d5c09012SAndroid Build Coastguard Worker// `text_sentiment_2.jsonl`,...,`text_sentiment_N.jsonl` 1122*d5c09012SAndroid Build Coastguard Worker// will be created, where N may be 1, and depends on the 1123*d5c09012SAndroid Build Coastguard Worker// total number of inputs and annotations found. 1124*d5c09012SAndroid Build Coastguard Worker// 1125*d5c09012SAndroid Build Coastguard Worker// Each .JSONL file will contain, per line, a JSON representation of a 1126*d5c09012SAndroid Build Coastguard Worker// proto that wraps input text file (or document) in 1127*d5c09012SAndroid Build Coastguard Worker// the text snippet (or document) proto and a list of 1128*d5c09012SAndroid Build Coastguard Worker// zero or more AnnotationPayload protos (called annotations), which 1129*d5c09012SAndroid Build Coastguard Worker// have text_sentiment detail populated. A single text file (or 1130*d5c09012SAndroid Build Coastguard Worker// document) will be listed only once with all its annotations, and its 1131*d5c09012SAndroid Build Coastguard Worker// annotations will never be split across files. 1132*d5c09012SAndroid Build Coastguard Worker// 1133*d5c09012SAndroid Build Coastguard Worker// If prediction for any input file (or document) failed (partially or 1134*d5c09012SAndroid Build Coastguard Worker// completely), then additional `errors_1.jsonl`, `errors_2.jsonl`,..., 1135*d5c09012SAndroid Build Coastguard Worker// `errors_N.jsonl` files will be created (N depends on total number of 1136*d5c09012SAndroid Build Coastguard Worker// failed predictions). These files will have a JSON representation of a 1137*d5c09012SAndroid Build Coastguard Worker// proto that wraps input file followed by exactly one 1138*d5c09012SAndroid Build Coastguard Worker// [`google.rpc.Status`](https://github.com/googleapis/googleapis/blob/master/google/rpc/status.proto) 1139*d5c09012SAndroid Build Coastguard Worker// containing only `code` and `message`. 1140*d5c09012SAndroid Build Coastguard Worker// 1141*d5c09012SAndroid Build Coastguard Worker// * For Text Extraction: 1142*d5c09012SAndroid Build Coastguard Worker// In the created directory files `text_extraction_1.jsonl`, 1143*d5c09012SAndroid Build Coastguard Worker// `text_extraction_2.jsonl`,...,`text_extraction_N.jsonl` 1144*d5c09012SAndroid Build Coastguard Worker// will be created, where N may be 1, and depends on the 1145*d5c09012SAndroid Build Coastguard Worker// total number of inputs and annotations found. 1146*d5c09012SAndroid Build Coastguard Worker// The contents of these .JSONL file(s) depend on whether the input 1147*d5c09012SAndroid Build Coastguard Worker// used inline text, or documents. 1148*d5c09012SAndroid Build Coastguard Worker// If input was inline, then each .JSONL file will contain, per line, 1149*d5c09012SAndroid Build Coastguard Worker// a JSON representation of a proto that wraps given in request text 1150*d5c09012SAndroid Build Coastguard Worker// snippet's "id" (if specified), followed by input text snippet, 1151*d5c09012SAndroid Build Coastguard Worker// and a list of zero or more 1152*d5c09012SAndroid Build Coastguard Worker// AnnotationPayload protos (called annotations), which have 1153*d5c09012SAndroid Build Coastguard Worker// text_extraction detail populated. A single text snippet will be 1154*d5c09012SAndroid Build Coastguard Worker// listed only once with all its annotations, and its annotations will 1155*d5c09012SAndroid Build Coastguard Worker// never be split across files. 1156*d5c09012SAndroid Build Coastguard Worker// If input used documents, then each .JSONL file will contain, per 1157*d5c09012SAndroid Build Coastguard Worker// line, a JSON representation of a proto that wraps given in request 1158*d5c09012SAndroid Build Coastguard Worker// document proto, followed by its OCR-ed representation in the form 1159*d5c09012SAndroid Build Coastguard Worker// of a text snippet, finally followed by a list of zero or more 1160*d5c09012SAndroid Build Coastguard Worker// AnnotationPayload protos (called annotations), which have 1161*d5c09012SAndroid Build Coastguard Worker// text_extraction detail populated and refer, via their indices, to 1162*d5c09012SAndroid Build Coastguard Worker// the OCR-ed text snippet. A single document (and its text snippet) 1163*d5c09012SAndroid Build Coastguard Worker// will be listed only once with all its annotations, and its 1164*d5c09012SAndroid Build Coastguard Worker// annotations will never be split across files. 1165*d5c09012SAndroid Build Coastguard Worker// If prediction for any text snippet failed (partially or completely), 1166*d5c09012SAndroid Build Coastguard Worker// then additional `errors_1.jsonl`, `errors_2.jsonl`,..., 1167*d5c09012SAndroid Build Coastguard Worker// `errors_N.jsonl` files will be created (N depends on total number of 1168*d5c09012SAndroid Build Coastguard Worker// failed predictions). These files will have a JSON representation of a 1169*d5c09012SAndroid Build Coastguard Worker// proto that wraps either the "id" : "<id_value>" (in case of inline) 1170*d5c09012SAndroid Build Coastguard Worker// or the document proto (in case of document) but here followed by 1171*d5c09012SAndroid Build Coastguard Worker// exactly one 1172*d5c09012SAndroid Build Coastguard Worker// [`google.rpc.Status`](https://github.com/googleapis/googleapis/blob/master/google/rpc/status.proto) 1173*d5c09012SAndroid Build Coastguard Worker// containing only `code` and `message`. 1174*d5c09012SAndroid Build Coastguard Worker// 1175*d5c09012SAndroid Build Coastguard Worker// * For Tables: 1176*d5c09012SAndroid Build Coastguard Worker// Output depends on whether 1177*d5c09012SAndroid Build Coastguard Worker// [gcs_destination][google.cloud.automl.v1p1beta.BatchPredictOutputConfig.gcs_destination] 1178*d5c09012SAndroid Build Coastguard Worker// or 1179*d5c09012SAndroid Build Coastguard Worker// [bigquery_destination][google.cloud.automl.v1p1beta.BatchPredictOutputConfig.bigquery_destination] 1180*d5c09012SAndroid Build Coastguard Worker// is set (either is allowed). 1181*d5c09012SAndroid Build Coastguard Worker// Google Cloud Storage case: 1182*d5c09012SAndroid Build Coastguard Worker// In the created directory files `tables_1.csv`, `tables_2.csv`,..., 1183*d5c09012SAndroid Build Coastguard Worker// `tables_N.csv` will be created, where N may be 1, and depends on 1184*d5c09012SAndroid Build Coastguard Worker// the total number of the successfully predicted rows. 1185*d5c09012SAndroid Build Coastguard Worker// For all CLASSIFICATION 1186*d5c09012SAndroid Build Coastguard Worker// [prediction_type-s][google.cloud.automl.v1p1beta.TablesModelMetadata.prediction_type]: 1187*d5c09012SAndroid Build Coastguard Worker// Each .csv file will contain a header, listing all columns' 1188*d5c09012SAndroid Build Coastguard Worker// [display_name-s][google.cloud.automl.v1p1beta.ColumnSpec.display_name] 1189*d5c09012SAndroid Build Coastguard Worker// given on input followed by M target column names in the format of 1190*d5c09012SAndroid Build Coastguard Worker// "<[target_column_specs][google.cloud.automl.v1p1beta.TablesModelMetadata.target_column_spec] 1191*d5c09012SAndroid Build Coastguard Worker// [display_name][google.cloud.automl.v1p1beta.ColumnSpec.display_name]>_<target 1192*d5c09012SAndroid Build Coastguard Worker// value>_score" where M is the number of distinct target values, 1193*d5c09012SAndroid Build Coastguard Worker// i.e. number of distinct values in the target column of the table 1194*d5c09012SAndroid Build Coastguard Worker// used to train the model. Subsequent lines will contain the 1195*d5c09012SAndroid Build Coastguard Worker// respective values of successfully predicted rows, with the last, 1196*d5c09012SAndroid Build Coastguard Worker// i.e. the target, columns having the corresponding prediction 1197*d5c09012SAndroid Build Coastguard Worker// [scores][google.cloud.automl.v1p1beta.TablesAnnotation.score]. 1198*d5c09012SAndroid Build Coastguard Worker// For REGRESSION and FORECASTING 1199*d5c09012SAndroid Build Coastguard Worker// [prediction_type-s][google.cloud.automl.v1p1beta.TablesModelMetadata.prediction_type]: 1200*d5c09012SAndroid Build Coastguard Worker// Each .csv file will contain a header, listing all columns' 1201*d5c09012SAndroid Build Coastguard Worker// [display_name-s][google.cloud.automl.v1p1beta.display_name] 1202*d5c09012SAndroid Build Coastguard Worker// given on input followed by the predicted target column with name 1203*d5c09012SAndroid Build Coastguard Worker// in the format of 1204*d5c09012SAndroid Build Coastguard Worker// "predicted_<[target_column_specs][google.cloud.automl.v1p1beta.TablesModelMetadata.target_column_spec] 1205*d5c09012SAndroid Build Coastguard Worker// [display_name][google.cloud.automl.v1p1beta.ColumnSpec.display_name]>" 1206*d5c09012SAndroid Build Coastguard Worker// Subsequent lines will contain the respective values of 1207*d5c09012SAndroid Build Coastguard Worker// successfully predicted rows, with the last, i.e. the target, 1208*d5c09012SAndroid Build Coastguard Worker// column having the predicted target value. 1209*d5c09012SAndroid Build Coastguard Worker// If prediction for any rows failed, then an additional 1210*d5c09012SAndroid Build Coastguard Worker// `errors_1.csv`, `errors_2.csv`,..., `errors_N.csv` will be 1211*d5c09012SAndroid Build Coastguard Worker// created (N depends on total number of failed rows). These files 1212*d5c09012SAndroid Build Coastguard Worker// will have analogous format as `tables_*.csv`, but always with a 1213*d5c09012SAndroid Build Coastguard Worker// single target column having 1214*d5c09012SAndroid Build Coastguard Worker// [`google.rpc.Status`](https://github.com/googleapis/googleapis/blob/master/google/rpc/status.proto) 1215*d5c09012SAndroid Build Coastguard Worker// represented as a JSON string, and containing only `code` and 1216*d5c09012SAndroid Build Coastguard Worker// `message`. 1217*d5c09012SAndroid Build Coastguard Worker// BigQuery case: 1218*d5c09012SAndroid Build Coastguard Worker// [bigquery_destination][google.cloud.automl.v1p1beta.OutputConfig.bigquery_destination] 1219*d5c09012SAndroid Build Coastguard Worker// pointing to a BigQuery project must be set. In the given project a 1220*d5c09012SAndroid Build Coastguard Worker// new dataset will be created with name 1221*d5c09012SAndroid Build Coastguard Worker// `prediction_<model-display-name>_<timestamp-of-prediction-call>` 1222*d5c09012SAndroid Build Coastguard Worker// where <model-display-name> will be made 1223*d5c09012SAndroid Build Coastguard Worker// BigQuery-dataset-name compatible (e.g. most special characters will 1224*d5c09012SAndroid Build Coastguard Worker// become underscores), and timestamp will be in 1225*d5c09012SAndroid Build Coastguard Worker// YYYY_MM_DDThh_mm_ss_sssZ "based on ISO-8601" format. In the dataset 1226*d5c09012SAndroid Build Coastguard Worker// two tables will be created, `predictions`, and `errors`. 1227*d5c09012SAndroid Build Coastguard Worker// The `predictions` table's column names will be the input columns' 1228*d5c09012SAndroid Build Coastguard Worker// [display_name-s][google.cloud.automl.v1p1beta.ColumnSpec.display_name] 1229*d5c09012SAndroid Build Coastguard Worker// followed by the target column with name in the format of 1230*d5c09012SAndroid Build Coastguard Worker// "predicted_<[target_column_specs][google.cloud.automl.v1p1beta.TablesModelMetadata.target_column_spec] 1231*d5c09012SAndroid Build Coastguard Worker// [display_name][google.cloud.automl.v1p1beta.ColumnSpec.display_name]>" 1232*d5c09012SAndroid Build Coastguard Worker// The input feature columns will contain the respective values of 1233*d5c09012SAndroid Build Coastguard Worker// successfully predicted rows, with the target column having an 1234*d5c09012SAndroid Build Coastguard Worker// ARRAY of 1235*d5c09012SAndroid Build Coastguard Worker// [AnnotationPayloads][google.cloud.automl.v1p1beta.AnnotationPayload], 1236*d5c09012SAndroid Build Coastguard Worker// represented as STRUCT-s, containing 1237*d5c09012SAndroid Build Coastguard Worker// [TablesAnnotation][google.cloud.automl.v1p1beta.TablesAnnotation]. 1238*d5c09012SAndroid Build Coastguard Worker// The `errors` table contains rows for which the prediction has 1239*d5c09012SAndroid Build Coastguard Worker// failed, it has analogous input columns while the target column name 1240*d5c09012SAndroid Build Coastguard Worker// is in the format of 1241*d5c09012SAndroid Build Coastguard Worker// "errors_<[target_column_specs][google.cloud.automl.v1p1beta.TablesModelMetadata.target_column_spec] 1242*d5c09012SAndroid Build Coastguard Worker// [display_name][google.cloud.automl.v1p1beta.ColumnSpec.display_name]>", 1243*d5c09012SAndroid Build Coastguard Worker// and as a value has 1244*d5c09012SAndroid Build Coastguard Worker// [`google.rpc.Status`](https://github.com/googleapis/googleapis/blob/master/google/rpc/status.proto) 1245*d5c09012SAndroid Build Coastguard Worker// represented as a STRUCT, and containing only `code` and `message`. 1246*d5c09012SAndroid Build Coastguard Workermessage BatchPredictOutputConfig { 1247*d5c09012SAndroid Build Coastguard Worker // The destination of the output. 1248*d5c09012SAndroid Build Coastguard Worker oneof destination { 1249*d5c09012SAndroid Build Coastguard Worker // Required. The Google Cloud Storage location of the directory where the output is to 1250*d5c09012SAndroid Build Coastguard Worker // be written to. 1251*d5c09012SAndroid Build Coastguard Worker GcsDestination gcs_destination = 1 [(google.api.field_behavior) = REQUIRED]; 1252*d5c09012SAndroid Build Coastguard Worker } 1253*d5c09012SAndroid Build Coastguard Worker} 1254*d5c09012SAndroid Build Coastguard Worker 1255*d5c09012SAndroid Build Coastguard Worker// Output configuration for ModelExport Action. 1256*d5c09012SAndroid Build Coastguard Workermessage ModelExportOutputConfig { 1257*d5c09012SAndroid Build Coastguard Worker // The destination of the output. 1258*d5c09012SAndroid Build Coastguard Worker oneof destination { 1259*d5c09012SAndroid Build Coastguard Worker // Required. The Google Cloud Storage location where the model is to be written to. 1260*d5c09012SAndroid Build Coastguard Worker // This location may only be set for the following model formats: 1261*d5c09012SAndroid Build Coastguard Worker // "tflite", "edgetpu_tflite", "tf_saved_model", "tf_js", "core_ml". 1262*d5c09012SAndroid Build Coastguard Worker // 1263*d5c09012SAndroid Build Coastguard Worker // Under the directory given as the destination a new one with name 1264*d5c09012SAndroid Build Coastguard Worker // "model-export-<model-display-name>-<timestamp-of-export-call>", 1265*d5c09012SAndroid Build Coastguard Worker // where timestamp is in YYYY-MM-DDThh:mm:ss.sssZ ISO-8601 format, 1266*d5c09012SAndroid Build Coastguard Worker // will be created. Inside the model and any of its supporting files 1267*d5c09012SAndroid Build Coastguard Worker // will be written. 1268*d5c09012SAndroid Build Coastguard Worker GcsDestination gcs_destination = 1 [(google.api.field_behavior) = REQUIRED]; 1269*d5c09012SAndroid Build Coastguard Worker } 1270*d5c09012SAndroid Build Coastguard Worker 1271*d5c09012SAndroid Build Coastguard Worker // The format in which the model must be exported. The available, and default, 1272*d5c09012SAndroid Build Coastguard Worker // formats depend on the problem and model type (if given problem and type 1273*d5c09012SAndroid Build Coastguard Worker // combination doesn't have a format listed, it means its models are not 1274*d5c09012SAndroid Build Coastguard Worker // exportable): 1275*d5c09012SAndroid Build Coastguard Worker // 1276*d5c09012SAndroid Build Coastguard Worker // * For Image Classification mobile-low-latency-1, mobile-versatile-1, 1277*d5c09012SAndroid Build Coastguard Worker // mobile-high-accuracy-1: 1278*d5c09012SAndroid Build Coastguard Worker // "tflite" (default), "edgetpu_tflite", "tf_saved_model", "tf_js", 1279*d5c09012SAndroid Build Coastguard Worker // "docker". 1280*d5c09012SAndroid Build Coastguard Worker // 1281*d5c09012SAndroid Build Coastguard Worker // * For Image Classification mobile-core-ml-low-latency-1, 1282*d5c09012SAndroid Build Coastguard Worker // mobile-core-ml-versatile-1, mobile-core-ml-high-accuracy-1: 1283*d5c09012SAndroid Build Coastguard Worker // "core_ml" (default). 1284*d5c09012SAndroid Build Coastguard Worker // 1285*d5c09012SAndroid Build Coastguard Worker // * For Image Object Detection mobile-low-latency-1, mobile-versatile-1, 1286*d5c09012SAndroid Build Coastguard Worker // mobile-high-accuracy-1: 1287*d5c09012SAndroid Build Coastguard Worker // "tflite", "tf_saved_model", "tf_js". 1288*d5c09012SAndroid Build Coastguard Worker // Formats description: 1289*d5c09012SAndroid Build Coastguard Worker // 1290*d5c09012SAndroid Build Coastguard Worker // * tflite - Used for Android mobile devices. 1291*d5c09012SAndroid Build Coastguard Worker // * edgetpu_tflite - Used for [Edge TPU](https://cloud.google.com/edge-tpu/) 1292*d5c09012SAndroid Build Coastguard Worker // devices. 1293*d5c09012SAndroid Build Coastguard Worker // * tf_saved_model - A tensorflow model in SavedModel format. 1294*d5c09012SAndroid Build Coastguard Worker // * tf_js - A [TensorFlow.js](https://www.tensorflow.org/js) model that can 1295*d5c09012SAndroid Build Coastguard Worker // be used in the browser and in Node.js using JavaScript. 1296*d5c09012SAndroid Build Coastguard Worker // * docker - Used for Docker containers. Use the params field to customize 1297*d5c09012SAndroid Build Coastguard Worker // the container. The container is verified to work correctly on 1298*d5c09012SAndroid Build Coastguard Worker // ubuntu 16.04 operating system. See more at 1299*d5c09012SAndroid Build Coastguard Worker // [containers 1300*d5c09012SAndroid Build Coastguard Worker // quickstart](https://cloud.google.com/vision/automl/docs/containers-gcs-quickstart) 1301*d5c09012SAndroid Build Coastguard Worker // * core_ml - Used for iOS mobile devices. 1302*d5c09012SAndroid Build Coastguard Worker string model_format = 4; 1303*d5c09012SAndroid Build Coastguard Worker 1304*d5c09012SAndroid Build Coastguard Worker // Additional model-type and format specific parameters describing the 1305*d5c09012SAndroid Build Coastguard Worker // requirements for the to be exported model files, any string must be up to 1306*d5c09012SAndroid Build Coastguard Worker // 25000 characters long. 1307*d5c09012SAndroid Build Coastguard Worker // 1308*d5c09012SAndroid Build Coastguard Worker // * For `docker` format: 1309*d5c09012SAndroid Build Coastguard Worker // `cpu_architecture` - (string) "x86_64" (default). 1310*d5c09012SAndroid Build Coastguard Worker // `gpu_architecture` - (string) "none" (default), "nvidia". 1311*d5c09012SAndroid Build Coastguard Worker map<string, string> params = 2; 1312*d5c09012SAndroid Build Coastguard Worker} 1313*d5c09012SAndroid Build Coastguard Worker 1314*d5c09012SAndroid Build Coastguard Worker// The Google Cloud Storage location for the input content. 1315*d5c09012SAndroid Build Coastguard Workermessage GcsSource { 1316*d5c09012SAndroid Build Coastguard Worker // Required. Google Cloud Storage URIs to input files, up to 2000 1317*d5c09012SAndroid Build Coastguard Worker // characters long. Accepted forms: 1318*d5c09012SAndroid Build Coastguard Worker // * Full object path, e.g. gs://bucket/directory/object.csv 1319*d5c09012SAndroid Build Coastguard Worker repeated string input_uris = 1 [(google.api.field_behavior) = REQUIRED]; 1320*d5c09012SAndroid Build Coastguard Worker} 1321*d5c09012SAndroid Build Coastguard Worker 1322*d5c09012SAndroid Build Coastguard Worker// The Google Cloud Storage location where the output is to be written to. 1323*d5c09012SAndroid Build Coastguard Workermessage GcsDestination { 1324*d5c09012SAndroid Build Coastguard Worker // Required. Google Cloud Storage URI to output directory, up to 2000 1325*d5c09012SAndroid Build Coastguard Worker // characters long. 1326*d5c09012SAndroid Build Coastguard Worker // Accepted forms: 1327*d5c09012SAndroid Build Coastguard Worker // * Prefix path: gs://bucket/directory 1328*d5c09012SAndroid Build Coastguard Worker // The requesting user must have write permission to the bucket. 1329*d5c09012SAndroid Build Coastguard Worker // The directory is created if it doesn't exist. 1330*d5c09012SAndroid Build Coastguard Worker string output_uri_prefix = 1 [(google.api.field_behavior) = REQUIRED]; 1331*d5c09012SAndroid Build Coastguard Worker} 1332