v1 and v2 are incompatible when loading local libraries.

[lian-yue]

GitHub repository with your minimal reproducible example (do not leave this field blank or fill out this field with "github.com/bufbuild/buf" or we will automatically close your issue, see the instructions above!)

https://github.com/lian-yue/bufver

Commands

./gen.v2.sh

Output

buf dep update: ./
buf dep update: ./pkg/type
buf generate: ./pkg/type
buf dep update: ./pkg/error
buf generate: ./pkg/error
buf dep update: ./mod/user
Failure: resource with name "bufver/error" was not found
buf generate: ./mod/user
proto/userpb/user.proto:4:8:import "typepb/date.proto": file does not exist
Code has generated

Expected Output

buf dep update: ./
buf dep update: ./pkg/type
buf generate: ./pkg/type
buf dep update: ./pkg/error
buf generate: ./pkg/error
buf dep update: ./mod/user
buf generate: ./mod/user
Code has generated

Anything else?

When the parent directory has a buf.yaml file configured with the modules field, for example:

/buf.yaml

version: v2
modules:
  - path: pkg/type/proto
    name: buf.build/bufver/type
  - path: pkg/error/proto
    name: buf.build/bufver/error
  - path: mod/user/proto
    name: buf.build/bufver/user
  - path: mod/post/proto
    name: buf.build/bufver/post
deps:
  - buf.build/bufbuild/protovalidate
lint:
  use:
    - STANDARD
breaking:
  use:
    - FILE

In version v1, local files can be loaded without errors.
In version v2, errors occur indicating that the file cannot be found.

Configuration that does not cause errors in version v1:

/mod/post/buf.yaml

version: v1
name: buf.build/bufver/post
deps:
  - buf.build/bufbuild/protovalidate
  - buf.build/bufver/type
  - buf.build/bufver/error
lint:
  use:
    - STANDARD
breaking:
  use:
    - FILE

Configuration in version v2:

/mod/post/buf.yaml

version: v2
modules:
  - path: proto
    name: buf.build/bufver/post
deps:
  - buf.build/bufver/type
  - buf.build/bufver/error
breaking:
  use:
    - FILE
lint:
  use:
    - STANDARD

This configuration causes errors.

I want the parent configuration with path + name fields to prioritize loading local files instead of directly loading remote file contents. This is because the local setup is intended to manage multiple pkg directories.

The /mod/post/buf.yaml file's modules.path field cannot be configured as ../../pkg/post.

Reproducing the Issue: Version 2 Cannot Execute Successfully

./gen.v2.sh

Version 1 Can Execute Successfully

./gen.v1.sh

[emcfarlane]

Hi @lian-yue , this looks like a configuration issue when migrating multiple modules to v2. I'll propose a solution that moves all the buf.yaml files to the root as a single workspace, then with multiple generate templates to output generated code to its current location. More details can be found in the migration guide.

The top buf.yaml folder should now contain the directory of each submodules, remove the buf.yaml and buf.lock file from the child modules:

version: v2
modules:
  - path: mod/post/proto  # Directory path is changed to the `proto` directory.
    name: buf.build/bufver/post
  - path: pkg/error/proto
    name: buf.build/bufver/error
  - path: pkg/type/proto
    name: buf.build/bufver/type
deps:
  - buf.build/bufbuild/protovalidate. # Only external non-local deps need to be specified
lint:
  use:
    - STANDARD
  except:
    - FIELD_NOT_REQUIRED
    - PACKAGE_NO_IMPORT_CYCLE
  disallow_comment_ignores: true
breaking:
  use:
    - FILE
  except:
    - EXTENSION_NO_DELETE
    - FIELD_SAME_DEFAULT

Now to output generated code in each modules directory you can use multiple buf.gen.yaml tailored for each, and run in the root. We will specify the input parameter to generate only for that specific module. Here is one example for the buf.gen.error.yaml file for the buf.builld/bufver/error module:

version: v2
managed:
  enabled: true
  disable:
    - file_option: go_package
      module: buf.build/bufver/type
  override:
    - file_option: go_package_prefix
      value: github.com/lian-yue/bufver/pkg/error/genpb/go
plugins:
  - local: protoc-gen-go
    opt: paths=source_relative 
    out: ./pkg/error/genpb/go  # Directory to output
inputs:
  - directory: ./pkg/error/proto # Input is specified so only this module is generated.

To now generate this module for the output in ./pkg/error/genpb/go run in the root of the project:

 buf generate --template=buf.gen.error.yaml 

This can be done for each module, invoking buf generate for each template. This keeps the current code output the same, although a simplification would be to group the output of the different code generations into a single buf.gen.yaml and one location would now be possible too.

[lian-yue]

@emcfarlane

Currently, I can avoid migrating the submodule's buf.yaml file as long as it remains in version 1, which will still be compatible. The buf.gen.yaml file can use version 2.

What I hope to achieve is that each repository is independent, with the root directory only configuring each repository's name and path, rather than running buf generate in the root directory to generate files.

If everything is configured in the root directory, then it essentially becomes a single repository.

[emcfarlane]

The error: proto/userpb/user.proto:4:8:import "typepb/date.proto": file does not exist is due to no buf.lock file exising in the directory ./mod/user. You must run buf dep update to ensure the dependencies are resolved before building.

[emcfarlane]

Closing as I think this issue is resolved. Please re-open if there is an issue or reach out on our public slack to get any help.