Private class fields

[skberkeley]

This PR introduces private fields for classes as a PR against the mega RFC (#242).

We introduce a different syntax than the last private field PR, emphasize the issues with having just a private keyword, and de-emphasize performance/implementation considerations.

Enjoy!

[TenebrisNoctua]

Honestly, this is so much better than the previous RFC, but still some problem points:

• I believe that instances of a class containing the private fields of the parent class is a bad solution.

Consider this example:

open class Point
    private _x: number
    private _y: number

    function __eq(self: Point, other)
        return self._x == other._x and self._y == other._y
    end
end

class ThreeDPoint extends Point
    private _z: number
end

This should normally fail. _x and _y should not remain on instances of ThreeDPoint, private should always mean private to a specific class, and never to any other class.

Classes are already opt-in to inheritance, so we can use protected for this instead:

open class Point
    protected x: number
    protected y: number

    function __eq(self: Point, other)
        return self.x == other.x and self.y == other.y
    end
end

class ThreeDPoint extends Point
    private _z: number
end

• Couldn't really understand from the RFC on what we're planning to do with the ability to not include an access specifier keyword for POD-like classes like this:

class Vector
    x: number
    y: number
end

We assume these properties are all public, but if we introduce a new field with an access specifier different than public, it creates ambiguity.

This could perhaps be solved with a parse-error:

class Vector -- Error: Must explicitly define every field with an access specifier
    x: number
    y: number
    private z: number
end

• Since we know that a class' private fields can never be accessed by its derived classes, as that behavior is reserved for protected, in that same fork, this becomes less of an ambiguity:

open class Base
    private x: number

    function compare(self, other)
        return self.x == other.x
    end
end

class Derived extends Base
    public x: number
end

local a = Base.new(...)
local b = Derived.new(...)

-- We can know that this will compare Base.x to Derived.x.
local res = a:compare(b)

In this case, we can know that Derived will never have the private field of Base, and thus compare method compares the private field of Base to the public field of Derived.

[skberkeley]

Thank your feedback! I will do my best to address all your points, but please let me know if I miss any!

• This may come down to a matter of opinion, but we firmly believe that child class instances should contain the private fields of their parents. This is the norm for other object oriented languages, and is generally a key part of the abstraction barrier provided by inheritance.
  More concretely, if I am the author of the Point class, all I should have to care about when you compare an instance of Point is that you compare to another instance of Point. This comparison should work whether you compare to ThreeDPoint or any other class that has Point in its inheritance chain. For this to work, instances of these deriving classes need to contain the private fields of Point, even if the child classes themselves don't have a way to access them.
• I can add a clarification for this, but all fields must be declared with an access modifier (either public or private for now).

[TenebrisNoctua]

Thank your feedback! I will do my best to address all your points, but please let me know if I miss any!

• This may come down to a matter of opinion, but we firmly believe that child class instances should contain the private fields of their parents. This is the norm for other object oriented languages, and is generally a key part of the abstraction barrier provided by inheritance.
  More concretely, if I am the author of the Point class, all I should have to care about when you compare an instance of Point is that you compare to another instance of Point. This comparison should work whether you compare to ThreeDPoint or any other class that has Point in its inheritance chain. For this to work, instances of these deriving classes need to contain the private fields of Point, even if the child classes themselves don't have a way to access them.

You can achieve the same exact system using protected. No one is stopping you there. It's just a matter of explicitness, separating this behavior to make it explicit using protected resolves the ambiguity, and allows you to not reserve any kind of glyph for any field.

It also allows the derived class to gain control over every one of its fields, which is what should be. Adding hidden fields will confuse users, and will create weird scenarios just like the one we're trying to solve in this RFC.

[TenebrisNoctua]

• I can add a clarification for this, but all fields must be declared with an access modifier (either public or private for now).

Yeah I don't mind this to be honest. Maybe some folks who like POD-classes may disagree, and perhaps a solution like in the fork I've mentioned can be made, but for now a restriction seems fine.

[andyfriesen]

This should normally fail. _x and _y should not remain on instances of ThreeDPoint, private should always mean private to a specific class, and never to any other class.

This proposal works nearly identically to how it works in C++, Java, and everything that's flowed from that. If private fields weren't available (but hidden) in derived classes, inheritance wouldn't even work for any class that has a private field.

open class BasePoint
    private _x: number
    private _y: number
    function __init(self, x, y) ... end
    function length(self)
        return math.sqrt(self._x * self._x + self._y * self._y)
    end
end

class NamedPoint extends BasePoint
    public name: string
    function __init(self, x, y, name) ... end
end

local np = NamedPoint.new(..)
print(np:length()) -- how could this work if np didn't have _x or _y?

[TenebrisNoctua]

This should normally fail. _x and _y should not remain on instances of ThreeDPoint, private should always mean private to a specific class, and never to any other class.

This proposal works nearly identically to how it works in C++, Java, and everything that's flowed from that. If private fields weren't available (but hidden) in derived classes, inheritance wouldn't even work for any class that has a private field.

open class BasePoint
    private _x: number
    private _y: number
    function __init(self, x, y) ... end
    function length(self)
        return math.sqrt(self._x * self._x + self._y * self._y)
    end
end

class NamedPoint extends BasePoint
    public name: string
    function __init(self, x, y, name) ... end
end

local np = NamedPoint.new(..)
print(np:length()) -- how could this work if np didn't have _x or _y?

With protected. This example would work quite well with it.

Personally, I don't complicate things by wanting to create a hidden field, to me, it is a worthless workaround for the fragile base class problem. In almost all of the cases I desired to use a hidden field, I only saw people (especially in Roblox codebases!) to find workarounds to continue violating it anyway.

[InfraredGodYT]

With protected. This example would work quite well with it.

Personally, I don't complicate things by wanting to create a hidden field, to me, it is a worthless workaround for the fragile base class problem. In almost all of the cases I desired to use a hidden field, I only saw people (especially in Roblox codebases!) to find workarounds to continue violating it anyway.

I agree with Noctua here. I also believe that instances of a class containing the private fields of the parent class is a bad idea when protected is usually used in this case. Inheritance without protected only amplifies the dissent against adding it to Luau in the first place as opposed to simply supporting composition.

[nnullcolumn]

Inheritance without protected only amplifies the dissent against adding it to Luau in the first place as opposed to simply supporting composition.

what does 'simply supporting composition' mean to you?

[InfraredGodYT]

Inheritance without protected only amplifies the dissent against adding it to Luau in the first place as opposed to simply supporting composition.

what does 'simply supporting composition' mean to you?

Allowing class fields that are instances/objects of other classes and not supporting inheritance?

[InfraredGodYT]

I also believe that instances of a class containing the private fields of the parent class is a bad idea when protected is usually used in this case.

I misinterpreted what was going on in this RFC and thought private was going to function like protected does in other languages. My mistake.

[Kenji-Shore]

This should normally fail. _x and _y should not remain on instances of ThreeDPoint, private should always mean private to a specific class, and never to any other class.

This proposal works nearly identically to how it works in C++, Java, and everything that's flowed from that. If private fields weren't available (but hidden) in derived classes, inheritance wouldn't even work for any class that has a private field.

open class BasePoint
    private _x: number
    private _y: number
    function __init(self, x, y) ... end
    function length(self)
        return math.sqrt(self._x * self._x + self._y * self._y)
    end
end

class NamedPoint extends BasePoint
    public name: string
    function __init(self, x, y, name) ... end
end

local np = NamedPoint.new(..)
print(np:length()) -- how could this work if np didn't have _x or _y?

With protected. This example would work quite well with it.

Personally, I don't complicate things by wanting to create a hidden field, to me, it is a worthless workaround for the fragile base class problem. In almost all of the cases I desired to use a hidden field, I only saw people (especially in Roblox codebases!) to find workarounds to continue violating it anyway.

With what you are saying, wouldn't this mean that private fields would only be allowed to be accessed through private methods (if the method wasn't private, then it would be inherited by derived classes, except those derived class instances do not contain the private base class fields, hence error)?

Also, what you are suggesting goes against the principle of encapsulation. Developers already use encapsulation in Lua with closures, so this is not anything new. Derived classes should not be able to manipulate data that is private to their base classes (because it creates opportunities for a derived class to interfere with its base class private fields in a way that the base class cannot anticipate). Base classes can offer methods to interface with its own private data and control the terms of the interaction.

[TenebrisNoctua]

This should normally fail. _x and _y should not remain on instances of ThreeDPoint, private should always mean private to a specific class, and never to any other class.

This proposal works nearly identically to how it works in C++, Java, and everything that's flowed from that. If private fields weren't available (but hidden) in derived classes, inheritance wouldn't even work for any class that has a private field.

open class BasePoint
    private _x: number
    private _y: number
    function __init(self, x, y) ... end
    function length(self)
        return math.sqrt(self._x * self._x + self._y * self._y)
    end
end

class NamedPoint extends BasePoint
    public name: string
    function __init(self, x, y, name) ... end
end

local np = NamedPoint.new(..)
print(np:length()) -- how could this work if np didn't have _x or _y?

With protected. This example would work quite well with it.
Personally, I don't complicate things by wanting to create a hidden field, to me, it is a worthless workaround for the fragile base class problem. In almost all of the cases I desired to use a hidden field, I only saw people (especially in Roblox codebases!) to find workarounds to continue violating it anyway.

With what you are saying, wouldn't this mean that private fields would only be allowed to be accessed through private methods (if the method wasn't private, then it would be inherited by derived classes, except those derived class instances do not contain the private base class fields, hence error)?

Also, what you are suggesting goes against the principle of encapsulation. Developers already use encapsulation in Lua with closures, so this is not anything new. Derived classes should not be able to manipulate data that is private to their base classes (because it creates opportunities for a derived class to interfere with its base class private fields in a way that the base class cannot anticipate). Base classes can offer methods to interface with its own private data and control the terms of the interaction.

Any field you'd need to use in that case would just be protected. Or you wouldn't make it protected, and just keep it to the class.

protected doesn't violate encapsulation. It's a matter of how you'd use it. I personally think that base class private fields existing under derived classes is not a healthy relationship in inheritance, from my personal experience.

A better argument would be to say that this behavior is quite the norm in many languages, and developers would expect to see it in Luau in one way or form. I propose it isn't necessary to make private contain this behavior, and we should remove it, or we implement it under a different access specifier to make it more explicit.

[Kenji-Shore]

protected doesn't violate encapsulation. It's a matter of how you'd use it. I personally think that base class private fields existing under derived classes is not a healthy relationship in inheritance, from my personal experience.

protected does violate encapsulation, though, in the sense that it allows for derived classes to modify base class fields in ways that the base class cannot anticipate. The whole point of encapsulation is allowing a class to provide a set interface and control access to its data. Getters and setters are a good example of this. There may be specific details of implementation that a base class wishes to keep tight control over, and does not want any derived classes to know or worry about. protected does not enable that, and with your suggestion, any functionality involving private would not work for inherited subclasses.