Discussion:
R10 priorities
(too old to reply)
j***@grid-sky.com
2015-12-26 19:57:09 UTC
Permalink
Raw Message
I'd love to start writing some M2 again. I'd argue that a Delphi2M2 converter or Golang2M2 converter would be great to encourage community interest. Given the compiler and basic libs are far enough along, if we just had an http module to start writing servers in I'd push for some degree of adoption in my organization right now. Maybe translating some code over from Delphi or Golang would provide some degree of a head start in this regard. Opinions?
Marco van de Voort
2015-12-26 21:39:16 UTC
Permalink
Raw Message
Post by j***@grid-sky.com
I'd love to start writing some M2 again. I'd argue that a Delphi2M2
converter or Golang2M2 converter would be great to encourage community
interest.
The languages are quite different. Of course you can always make a converter
that does a lot of the drudge work, but it won't be very high grade and
require a lot of manual work afterwards.

The change to unsigned arithmetic (and types) by default alone, and probably
the string handling too.
trijezdci
2015-12-26 23:14:54 UTC
Permalink
Raw Message
Post by Marco van de Voort
The change to unsigned arithmetic (and types) by default alone, and probably
the string handling too.
Modula-2 has always had both signed and unsigned types.

If you convert code from a language that has only got signed integers to a language that has got both signed and unsigned integers, you simply use only the signed integers and ignore unsigned integers.

It's the opposite direction that would cause extra work.

Besides, any type can be done in library in M2 R10. You want Pascal strings, write a library that conforms to blueprint ProtoString and it will behave as if it was built-in.

The hard work in a Delphi-to-M2R10 converter would be to convert variant records into an equivalent set of extensible records.

TYPE Variant = ( VarA, VarB, ... );

TYPE Foobar = RECORD
foo : Foo;
CASE v : Variant OF
VarA : bar : Bar
| VarB : baz : Baz
...
END
END

to

TYPE Foobar = RECORD ( NIL )
foo : Foo
END;

TYPE BarRec = RECORD ( Foobar )
bar : Bar
END;

TYPE BazRec = RECORD ( Foobar )
baz : Baz
END;

...

This may look straight forward on paper but it is quite a bit of work. Furthermore, when declaring variables of the type and using them, the Variant Record version will always use the Foobar type, whereas the Extensible Record will use a different type for each variant. This is also quite a bit of work.

At some point we will need a PIM4 to M2R10 translator and handle this. Since I already have a working front end for PIM now in the rewrite of M2C, it will make sense to use that and add an option to generate M2R10 as output.

If anyone wants to work on a Pascal/Delphi to M2R10 translator, I think it would probably be saving quite some effort if they simply wrote a front end that generates the same AST as M2C. M2C's middle and back end could then be used.

I have designed the AST so it covers both PIM and extended features, such as variant records and extensible records which have separate AST nodes. It is realised as an opaque library, with only a few functions and completely isolated from the parser.

That is to say, the parser does not have any knowledge of the AST internals, nor will any front end author writing for this AST need to know any internals. Architecturally it is front-end agnostic.

There are likely to be features in Delphi that don't map well to the existing AST nodes. However, thanks to the simplicity and modularity of the AST module, additional nodes could easily be added.

Here is the struct that implements the opaque AST node type

struct m2c_astnode_struct_t {
/* node_type */ m2c_ast_nodetype_t node_type;
/* lexeme */ m2c_string_t lexeme;
/* subnode_count */ uint_t subnode_count;
/* subnode_table */ m2c_astnode_t subnode_table[];
};



The node type is an enumeration, the subnode table is an array of pointers to the node type itself. Thanks to C's variadic argument lists and structs with flex array members, this array can be allocated exactly to the number of passed in subnodes.

Technically, there are two variants, even though I abstained from using variant structs (or unions as C calls them) one node-type for terminals (identifiers, number literals, character code literals and quoted literals) which uses the lexeme field only, and another for non-terminals (all else) which uses the subnode fields only. Avoiding the use of unions trades the space of one pointer in return for ease of coding and readability.


To add a new node type, all one has to do is add another value to the enumeration that defines node types and add parameters to an structured initialiser that says how many subnodes the new node type has and what the types of the subnodes shall be.

The new node type can then be created with the existing function

node = m2c_ast_new_node(MY_NEW_NODE_TYPE, subnode1, subnode2, ...);

Of course the tree walker needs to have a procedure that knows what to do with the new node type and the code generator will need a template to be populated from the new node type.

Yet, this is all a lot less work than writing your own AST, tree walker and code generator from scratch.

If the front end was to be written in Delphi, it should be possible to interface to the C libraries by some foreign function interfacing mechanism. If it was to be written in M2C, there will be a foreign function interface that allows M2 definition modules to be used for corresponding C implementations.

Note that the AST module is not yet in the public repo, I hope to be able to commit it within the next few days.

Anyway, this would be my recommendation if anyone wants to write a Delphi-M2R10 translator: Write a Delphi front-end (even with a subset initially) and hook into my AST library and back end for M2C to save yourself about half the work (or even more).

As for Golang, I reckon that the differences are more substantial than between Pascal and M2, and this would naturally cause more effort then. A Delphi front end would certainly be a good start and whoever does that will gain experience that would then come in handy for doing a Golang-M2R10 translator later.

Unfortunately, I cannot help on either translator (other than providing the AST library, tree walker and code generator) because I have to focus on the completion of the bootstrap and then the eventual compiler.

It would be worthwhile having though I think.
Marco van de Voort
2015-12-27 21:55:29 UTC
Permalink
Raw Message
Post by trijezdci
Besides, any type can be done in library in M2 R10. You want Pascal
strings, write a library that conforms to blueprint ProtoString and it
will behave as if it was built-in.
Is your variable length stringtype also such construct?

Personally what turned me off M2 for non embedded use was the very laborous
string support.
James Smith
2015-12-27 22:20:09 UTC
Permalink
Raw Message
And widestring/constant support will be some work that needs to be prioritized.
Marco van de Voort
2015-12-27 22:36:21 UTC
Permalink
Raw Message
Post by James Smith
And widestring/constant support will be some work that needs to be prioritized.
Not necessarily. Unicode is a difficult subject, and if you have non Windows
to regard too, copying the Windows/COM/Delphi utf16 model is not a given.

It is the hotly debated subject on the Lazarus/Free Pascal lists ever.
Marco van de Voort
2015-12-27 23:03:11 UTC
Permalink
Raw Message
Post by trijezdci
Modula-2 has always had both signed and unsigned types.
If you convert code from a language that has only got signed integers to a
language that has got both signed and unsigned integers, you simply use
only the signed integers and ignore unsigned integers.
That is a viable first order magnitude workaround, but VAR parameters still
have to match exactly, and combining parts (e.g. the Delphi code and RTL
functions) will result in signed+unsigned leading to widening, which is not
nice to have large scale.

I've roughly the same views as Chris. Limited automated conversion as
assistence and mostly for algorithmic code is viable. Large scale conversion
is hard, specially if the languages are not related (e.g. object models
don't map)
trijezdci
2015-12-28 09:31:42 UTC
Permalink
Raw Message
Post by Marco van de Voort
I've roughly the same views as Chris. Limited automated conversion as
assistence and mostly for algorithmic code is viable. Large scale conversion
is hard, specially if the languages are not related (e.g. object models
don't map)
I think you are misreading the motive by the original poster who said "encourage community interest".

In my view such translators are for all practical purposes USELESS, but as I said, I believe they are desirable to have. I have come to this view based on my own experience with the Logitech Turbo-Pascal to Modula-2 translator in the 1980s. I did not like it one bit. It was entirely useless to me.

HOWEVER, if it had not been for the fact that Logitech offered this translator, the company I worked for at the time would never have made the switch from Turbo-Pascal to Modula-2.

An open source translator would be even more likely to "encourage community interest" since anyone can in principle improve the performance of the translator when they find that something isn't translated in the way they thought it should be.

Yet, for the latter to have maximum effect it is perhaps best if the translator was written in the language it translates to, not in the language it translates from. After all the main purpose is not the product of translation, but getting people to engage with the revised language.
j***@gmail.com
2015-12-28 11:17:36 UTC
Permalink
Raw Message
In my experience, translating software from one source language in another is next to impossible. Even when the source and destination languages are similar, eg translating PC-9800 Basic into GW Basic. It was always faster and better to study the underlying 'problem' and then come up with a new algorithm. In many events the programmer of the original program did not know what he was doing anyway.

Translating higher level languages between each other, back and forth, is close to useless. It will generate a lot of nonsense code.

A program is not just a repetition of keywords. It is the mindset of a programmer at a given time in his own development path. I always found it very difficult to step into another man's head and figure out, 'why the hell did he do this, here?'.

So the chance that an automated program can do it effortlessly are minimal. If the OP wants to turn his software business from Delphi into Oberon/Modula-2 there is just one option: rewrite the full sources. Do not rely on 'older sources' as they also contain a lot of older thoughts plus the associated rework.

This is your major chance to do a full overhaul of your software libraries.
trijezdci
2015-12-28 13:13:24 UTC
Permalink
Raw Message
Post by j***@gmail.com
In my experience, translating software from one source language in another is next to impossible. Even when the source and destination languages are similar, eg translating PC-9800 Basic into GW Basic. It was always faster and better to study the underlying 'problem' and then come up with a new algorithm. In many events the programmer of the original program did not know what he was doing anyway.
This may be a very unpopular point of view, but ...

The very same applies to using somebody's code that is written in the same language you are using and having been written for and build with the same compiler you are using.

Unless the software was designed and written from the start with one of the primary priorities on maintainability by other people who may not even be able to talk to the original author when they have to maintain it, then the chances are that a complete rewrite is a better investment of your time than using that other person's code.
Post by j***@gmail.com
So the chance that an automated program can do it effortlessly are minimal. If the OP wants to turn his software business from Delphi into Oberon/Modula-2 there is just one option: rewrite the full sources. Do not rely on 'older sources' as they also contain a lot of older thoughts plus the associated rework.
I didn't think that was the OP's intention. He wrote "encourage community interest" which appears to be the motive. And yes, strange as it may seem, if there is such a translator, no matter how little use it is in practise, people will be more willing to engage with the target language.
Marco van de Voort
2015-12-28 15:12:29 UTC
Permalink
Raw Message
Post by trijezdci
Post by Marco van de Voort
I've roughly the same views as Chris. Limited automated conversion as
assistence and mostly for algorithmic code is viable. Large scale conversion
is hard, specially if the languages are not related (e.g. object models
don't map)
I think you are misreading the motive by the original poster who said
"encourage community interest".
I think the time can be better spent on something that actually yields
something for the users rather than superficial attraction.

Or are you already knee-deep into the manpower ?
Post by trijezdci
In my view such translators are for all practical purposes USELESS, but as
I said, I believe they are desirable to have. I have come to this view
based on my own experience with the Logitech Turbo-Pascal to Modula-2
translator in the 1980s. I did not like it one bit. It was entirely
useless to me.
Mostly it is yes. Sometimes I succesfully used C2pas for algorithmic code
(like compression), and header convertors have always been useful.

That would be the first thing I would imagine from a delphi2m2 project too.
Header conversion, since it is a source of (nearly) preprocessor-free
headers.
Post by trijezdci
Yet, for the latter to have maximum effect it is perhaps best if the
translator was written in the language it translates to, not in the
language it translates from.
I think so too. In general I think it is wise for the project to only invest
in M2 code, and not in other languages. Even for its own demands.

It is hard in the beginning but pays out long term.
Post by trijezdci
After all the main purpose is not the product of translation, but getting
people to engage with the revised language.
Translators usually target a subset, and higher level code often doesn't
translate well.

Header translators could be useful though.
James Smith
2015-12-28 16:16:54 UTC
Permalink
Raw Message
Post by Marco van de Voort
Header translators could be useful though.
Yes, and even just procedure boilerplate. Not expecting to process some code through a translator and expect it to run. Point is to give the developer a module starting point to understand what the translated lib does.
trijezdci
2015-12-28 17:53:25 UTC
Permalink
Raw Message
Post by Marco van de Voort
I think the time can be better spent on something that actually yields
something for the users rather than superficial attraction.
Quite probably.
Post by Marco van de Voort
Or are you already knee-deep into the manpower ?
No we're not, but we cannot stop volunteers from doing stuff they want to do.
Chris Burrows
2015-12-27 11:36:38 UTC
Permalink
Raw Message
Post by j***@grid-sky.com
I'd love to start writing some M2 again. I'd argue that a Delphi2M2 converter or Golang2M2 converter would be great to encourage community interest.
There are some Pascal to Modula-2 converters that have already been developed e.g. Logitech's Turbo Pascal to Modula-2 translator. The manual can be downloaded from bitsavers.org - it will give you some insights into what can and can't be easily done.

The source code of translators from Pascal to Oberon / Component Pascal is available. They could be easily adapted to target Modula-2 as well. They were originally adapted from Niklaus Wirth's Modula-2 to Oberon translator:

http://www.zinnamturm.eu/topics.htm#Translator

Both are strictly Standard Pascal or Turbo Pascal convertors - a convertor that could handle all of Delphi's Object Pascal constructs would be an insurmountable task.

In the past I have found such programming language translation tools only useful for converting small standalone algorithms. For anything more substantial it is better in the long run to re-implement from the original design specifications. If you don't have those - don't waste your time. Otherwise it's too much like trying to make an omelette from a scrambled egg.

Regards,
Chris Burrows

CFB Software
http://www.astrobe.com
James Smith
2015-12-27 18:59:24 UTC
Permalink
Raw Message
Agreed, especially in the case of Delphi, which IMO is an attempt to turn Pascal into C#. What's great about M2 is that the abstractions and generalizations are kept to a minimum, so you can keep more of the problem domain in your head. There's only so many ways you can do things - that's good IMO. Golang strikes me the same way as I use it more.

I've written lines and lines of both Delphi and golang, so a translator would still be helpful for me. What I'd propose is that ultimately the results of a golang to M2 translator would end up being a lot more edifying than a Delphi to M2 translator. Too much crap has been added to Delphi, but even 50% coverage might be helpful for me in some cases.

At any rate, there's obviously just not enough resources for M2 to mature into the general systems programming language with supporting community on par with other examples like Rust in a short amount of time. Focus is needed.

IMO nobody needs to be wasting time writing GTK bindings or porting to other OS's besides windows or Linux. In fact, just pick one OS and write a comprehensive, performant green thread http library for it so devs could get started writing web services. I know there is an embedded application interest for M2, but there's just not enough embedded work going on for this type of effort to suck down limited M2 community resources.

The reason why golang adoption has been so successful is that it's easy to pick up, keeps abstractions to a minimum, limits the number of ways things can be done, and so makes a wider range of dev skill levels productive more quickly. Well M2 has had that covered for a long time as an art form.

So, once the compiler is bringing joy, why not follow golang's lead and make internet development an initial sweet spot? We need to show how fast you can whip out a performant http server in M2, fast json and xml parsers, compression and encryption libraries. Until these libraries are mature, we can use golang packages starting with Go 1.5.

And back to translators - just part of a broader theme to give M2 a headstart and why I started this thread. Sorry for drifting.
Chris Burrows
2015-12-27 22:31:56 UTC
Permalink
Raw Message
Post by James Smith
Agreed, especially in the case of Delphi, which IMO is an attempt to turn Pascal into C#.
Not exactly. It is more a case of C# is an attempt to turn C into Delphi ;-)

The Delphi language (or Object Pascal as it was known at the time) was first released in 1995 - C# didn't appear until some years later in 2000. The lead architect of both was Anders Hejlsberg (who left Borland to join Microsoft in 1996) so it is not surprising there are some similarities.
Loading...