Join GitHub today
GitHub is home to over 20 million developers working together to host and review code, manage projects, and build software together.
reflect: resolve incompatibility between gc and gccgo for unexported embedded structs #12367
Comments
mikioh
added
the
Proposal
label
Aug 27, 2015
I think the statement here can be made much more precise. Here is my understanding. Today, given:
In the gc toolchain,
In the gccgo toolchain,
The fundamental problem here is that a Go program's source code can refer to t1.X (implicitly t1.t2.X) directly, while the reflect API cannot. The reflect API requires stepping into each field explicitly. The gc implementation allows reflection to set t1.X by recording the field t2 as exported (wrong): that makes it possible to set t1.t2.X (arguably right) but also all of t1.t2 (wrong). The gccgo implementation records t2 as unexported (right), which disallows setting t1.t2.X (arguably wrong) and also disallows setting t1.t2 (right). The proposal is that both implementations record t2 as unexported, and then change reflect to record whether a value like v2 is read-only only because it is an unexported embedded field. In that case, accessing an exported element within would clear the read-only bit. The effect would be that t2 is record as unexported (right), t1.t2.X is settable (arguably right), t1.t2 is not settable (right), and of course t1.t2.u is also not settable (right). That is, under this proposal:
That satisfies encoding/xml and encoding/json, disallows most of what is incorrectly allowed in the gc toolchain today, and allows what is arguably incorrectly disallowed in the gccgo toolchain today. Both encoding/json and encoding/xml assume today that if f.PkgPath != nil then the field is unexported and must be ignored. That test needs to be revised to f.PkgPath != nil && !f.Anonymous, so that they do try to walk into the embedded fields to look for unexported fields contained within. Any similar external packages would need the same ~1 line change. The new condition is correct both before and after the proposed change, so it could be made by any affected packages independent of the eventual Go 1.6 release. |
mwhudson
referenced this issue
Aug 28, 2015
Closed
cmd/compile: empty PkgPath for embedded, unexported struct field #7247
Thanks for the much clearer reformulation Russ. |
Preparations for json and xml package: See the actual change: Note that the tests added in the json and xml CLs will currently break for gccgo. Packages just checking for f.PkgPath to determine if a field is exported should implement a fix similar to those in 14011 and 14012 now to prevent breaking when CL 14010 is checked in. |
gopherbot
commented
Aug 28, 2015
CL https://golang.org/cl/14011 mentions this issue. |
gopherbot
commented
Aug 28, 2015
CL https://golang.org/cl/14012 mentions this issue. |
gopherbot
commented
Aug 28, 2015
CL https://golang.org/cl/14010 mentions this issue. |
gopherbot
commented
Aug 31, 2015
CL https://golang.org/cl/14085 mentions this issue. |
adg
added
Proposal
and removed
Proposal
labels
Sep 25, 2015
rsc
added this to the Proposal milestone
Oct 24, 2015
rsc
changed the title from
proposal: workable solution to reflect incompatibility between gc and gccgo
to
reflect: resolve incompatibility between gc and gccgo for unexported embedded structs
Oct 24, 2015
rsc
modified the milestones:
Go1.6Early,
Proposal
Oct 24, 2015
rsc
added
the
Proposal-Accepted
label
Oct 24, 2015
added a commit
that referenced
this issue
Oct 26, 2015
added a commit
that referenced
this issue
Oct 26, 2015
added a commit
that referenced
this issue
Oct 26, 2015
mpvl
closed this
in
afe9837
Oct 26, 2015
Code that assumes
means a field is unexported and must be ignored must now be revised to check for
for it to walk into the embedded structs to look for exported fields contained within. |
mpvl commentedAug 27, 2015
Current Situation
Both are not correct w.r.t. the way Go works: an unexported embedded struct should not be settable as a whole, however, exported fields and methods of unexported embedded structs may be promoted and be visible.
Goal
Solution
Make unexported embedded structs unsettable, but allow the reflect library to access exported fields and methods of such structs. Such fields could also be modified.
Rationale: this prevents blanket changes to values in structs that are generally not settable, while allowing access to fields that might be promoted.
The approach might still allow access to fields that are not accessible in the language, namely exported fields and methods that are not promoted because they are masked by or conflict with fields or methods of the same name. However, access to such fields is also necessary. Packages like encoding/xml and encoding/json allow users to map fields to alternative names using tags effectively breaking ties and making fields visible again. Disallowing access to blocked fields now would be too much of a disruptive change and break the Go 1 compatibility promise.
Impact
This will break packages like xml and json, but only minimally. The typical check for whether a field is exported, f.PkgPath != "", would have to be amended to f.PkgPath != "" && !f.Anonymous.
Alternatives
adopt gccgo behavior
Unacceptable: it would be too disruptive to packages like xml and json and would break documented behavior, breaking the Go 1 compatibility promise. It would mean reflect cannot give access to fields that are visible in Go itself.
adopt gc behavior
Less problematic, but it would make it very easy to access structs that are indicated as hidden. Users could bypass initialization code etc, raising various security issues. The proposed solution does not eliminate this, but it does considerably mitigate it by limiting access to fields that are deemed exported and settable if the embedded type were accessed directly.
package reflect provides a different API for accessing promoted fields and methods.
Provide a higher-level API for packages like xml and json for mapping structured data to struct fields
This would potentially give greater consistency and would make it easier for developers to write package like json and xml. However, it would be a huge change and similarly does not address the concerns of what to do with the current API. Like the current proposal, it would still allow access to non-promoted fields and methods. It could be implemented on top of the proposed solution.