aistudio real file upload

Training data often fails to load in Baidu AI Studio when Git LFS pointers are uploaded instead of raw content. This skill automates the detection and removal of LFS rules while safely uploading large datasets to PaddlePaddle repositories. It prevents the common "file not found" errors during training by ensuring repo files contain real bytes, not 100-byte pointers.

What it does

• Real-file verification ensures every upload contains actual data rather than Git LFS text pointers.
• LFS rule management identifies and removes conflicting .gitattributes rules that intercept training data.
• Atomic preflight checks validate all local sources and remote targets before committing any writes.
• Safe cleanup removes mistaken Windows backslash paths and stale directory prefixes using digest-bound manifests.
• README generation scaffolds dataset documentation using standardized templates for AI Studio.

How it works

1. Target Preflight: Provide a repo_id and local files; the tool scans the remote repository for existing objects and LFS rules.
2. Authorization Manifest: If a configuration change or deletion is required, the tool generates a local JSON manifest and a SHA-256 digest for review.
3. Explicit Confirmation: Run the command with the approved manifest digest and remote SHAs to apply repository changes.
4. Verified Upload: Upload files using forward-slash paths, followed by a post-upload size and content integrity check.

Frameworks & tools

Python scripts utilizing the Baidu AI Studio SDK and Git API. Designed for use with PaddlePaddle, ERNIE, and AI Studio dataset repositories.

Why this beats prompting it yourself

Standard prompting often ignores the nuance of the AI Studio Git API, leading to LFS pointer traps that break training pipelines. This skill enforces a strict preflight-and-authorization flow that prevents silent failures and accidental data corruption.

Use cases

• Uploading JSONL training data for MedRecord or GraphNorm fine-tuning.
• Fixing "trainDataFiles specified file does not exist" errors in AI Studio.
• Cleaning up accidentally uploaded Windows-style backslash paths in remote repos.
• Managing large dataset assets that exceed standard Git upload limits.

Known limitations

Requires an active AISTUDIO_ACCESS_TOKEN in the environment. Upload targets ending in slashes are rejected to prevent filename ambiguity.