What language idioms / paradigms / functions make it difficult to add support for "type providers"?

Question

F # 3.0 added provider type .

I wonder if it is possible to add this language function to other languages ​​that work on the CLR, for example C #, or if this function works only in a more functional / less OO programming style?

Answer 1

As Brian and Thomas emphasize, there is nothing particularly “functional” in this function. This is just a great way to provide metadata to the compiler.

The C # development team has long been using ideas like this. There was a proposal a few years before I joined the C # team for a function that would be called "type drawings" (or something like that), resulting in a combination of XML documents, XML schema, and custom code, offering type metadata is used by the C # compiler. I do not remember the details, and this obviously did not work. (Although it did affect the design and implementation of the Visual Studio Tools for Office document format I was working on at that time.)

In any case, we have no plans for the near horizon of adding such a function in C #, but we are watching with great interest whether it copes well with resolving customer problems in F #.

(As always, Eric reflects on the possible future features of unpublished and completely hypothetical products for entertainment purposes only.)

Answer 2

As Thomas says, it is theoretically easy to add this function to any statically typed language (although there is still a lot of work).

I am not a metaprogram expert, but @ SK-logic asks why a generic metaprogram compilation system is not used instead, and I will try to answer. I don't think you can easily achieve what you can do with F # type providers using metaprogramming, because F # type providers can be lazy and dynamically interactive during development. Let me give you an example that Don demonstrated in one of his early videos: Freebase . Freebase is like a schematized, programmable Wikipedia, it has data about everything. So you can write code line by line

for e in Freebase.Science.``Chemical Elements`` do printfn "%d: %s - %s" e.``Atomic number`` e.Name e.Discoverer.Name 

or something else (I don’t have the exact code), but it’s just as easy to write code that gets information about baseball statistics, or when famous actors were in drug rehabilitation facilities or in the form of millions of other types of information available through Freebase.

From the point of view of the implementation, it is impossible to create a schema for the whole Freebase and bring it to .NET a-priori; you cannot just take one step of compilation time at the beginning to install it all. You can do this for small data sources, and in fact, many other providers use this strategy, for example. an SQL type provider accesses the database and generates .NET types for all types in this database. But this strategy does not work for large cloud data warehouses such as Freebase, because there are too many interrelated types (if you tried to generate .NET metadata for all Freebase, you will find that there are so many millions types (one of which is ChemicalElement with AtomicNumber and Discoverer and Name and many other fields, but there are literally millions of these types) that you need more memory than is available for the 32-bit .NET process to represent the whole type scheme.

Thus, an F # type strategy is an API architecture that allows type providers to provide on-demand information that runs during development in the IDE. Until you type, for example. Freebase.Science. , a type provider does not need to know about objects under the categories of science, but as soon as you click . after Science , a provider of this type can go and request APIs to examine sibling general schemes to find out what categories exist in science, one of which is ChemicalElements . And then, trying to "insert" into one of them, he discovers that the elements have atomic numbers and what not. Thus, a type provider lazily extracts a fairly general scheme for processing the exact code that the user types into the editor at this point in time. As a result, the user still has the right to explore any part of the information universe, but any source file or interactive session will only study a small part of what is available. When the compilation time comes / codegen, the compiler only needs to generate enough code to precisely place the bits that the user actually used in his code, and not the potentially huge bits of the runtime to allow talking to the entire data store.

(Maybe you can do this with some of the modern metaprogramming tools now, I don’t know, but those that I learned about at school a long time ago could not easily handle this.)

Answer 3

I see no technical reason why something like type providers could not be added in C # or similar languages. The only langauges family that makes it difficult to add type providers (like F #) are languages ​​with dynamically typed languages.

F # type providers rely on the type information generated by the provider to be well distributed through the program, and the editor can use them to display useful IntelliSense. In dynamically typed languages, this will require more complex IDE support (and the "type providers" for dynamic languages ​​come down to just an IDE or IntelliSense).

Why are they implemented directly as an F # function? I think the metaprogramming system should be really complex (note that types are not actually created) to support this. Other things that could be done with this would not greatly affect the F # language (they will only make it too complicated, which is bad). However, you could get a similar thing if you had some kind of compiler extensibility.

In fact, I think this is how the C # team will add something like type providers in the future (they talked about compiler extensibility for some time).