C#: Move marshaling logic and generated glue to C#

We will be progressively moving most code to C#.
The plan is to only use Mono's embedding APIs to set things at launch.
This will make it much easier to later support CoreCLR too which
doesn't have rich embedding APIs.

Additionally the code in C# is more maintainable and makes it easier
to implement new features, e.g.: runtime codegen which we could use to
avoid using reflection for marshaling everytime a field, property or
method is accessed.

SOME NOTES ON INTEROP

We make the same assumptions as GDNative about the size of the Godot
structures we use. We take it a bit further by also assuming the layout
of fields in some cases, which is riskier but let's us squeeze out some
performance by avoiding unnecessary managed to native calls.

Code that deals with native structs is less safe than before as there's
no RAII and copy constructors in C#. It's like using the GDNative C API
directly. One has to take special care to free values they own.
Perhaps we could use roslyn analyzers to check this, but I don't know
any that uses attributes to determine what's owned or borrowed.

As to why we maily use pointers for native structs instead of ref/out:
- AFAIK (and confirmed with a benchmark) ref/out are pinned
  during P/Invoke calls and that has a cost.
- Native struct fields can't be ref/out in the first place.
- A `using` local can't be passed as ref/out, only `in`. Calling a
  method or property on an `in` value makes a silent copy, so we want
  to avoid `in`.

REGARDING THE BUILD SYSTEM

There's no longer a `mono_glue=yes/no` SCons options. We no longer
need to build with `mono_glue=no`, generate the glue and then build
again with `mono_glue=yes`. We build only once and generate the glue
(which is in C# now).
However, SCons no longer builds the C# projects for us. Instead one
must run `build_assemblies.py`, e.g.:
```sh
%godot_src_root%/modules/mono/build_scripts/build_assemblies.py \
        --godot-output-dir=%godot_src_root%/bin \
        --godot-target=release_debug`
```
We could turn this into a custom build target, but I don't know how
to do that with SCons (it's possible with Meson).

OTHER NOTES

Most of the moved code doesn't follow the C# naming convention and
still has the word Mono in the names despite no longer dealing with
Mono's embedding APIs. This is just temporary while transitioning,
to make it easier to understand what was moved where.

1 files changed, 28 insertions, 21 deletions

diff --git a/modules/mono/csharp_script.cpp b/modules/mono/csharp_script.cpp
index b95b63cf1f..2e6a6ef7e0 100644
--- a/modules/mono/csharp_script.cpp
+++ b/modules/mono/csharp_script.cpp
@@ -57,6 +57,7 @@
 #endif
 
 #include "godotsharp_dirs.h"
+#include "managed_callable.h"
 #include "mono_gd/gd_mono_cache.h"
 #include "mono_gd/gd_mono_class.h"
 #include "mono_gd/gd_mono_marshal.h"
@@ -108,6 +109,9 @@ Error CSharpLanguage::execute_file(const String &p_path) {
 	return OK;
 }
 
+extern void *godotsharp_pinvoke_funcs[101];
+[[maybe_unused]] volatile void **do_not_strip_godotsharp_pinvoke_funcs;
+
 void CSharpLanguage::init() {
 #ifdef DEBUG_METHODS_ENABLED
 	if (OS::get_singleton()->get_cmdline_args().find("--class-db-json")) {
@@ -118,20 +122,18 @@ void CSharpLanguage::init() {
 	}
 #endif
 
-	gdmono = memnew(GDMono);
-	gdmono->initialize();
+	// Hopefully this will be enough for all compilers. Otherwise we could use the printf on fake getenv trick.
+	do_not_strip_godotsharp_pinvoke_funcs = (volatile void **)godotsharp_pinvoke_funcs;
 
 #if defined(TOOLS_ENABLED) && defined(DEBUG_METHODS_ENABLED)
-	// Generate bindings here, before loading assemblies. 'initialize_load_assemblies' aborts
-	// the applications if the api assemblies or the main tools assembly is missing, but this
-	// is not a problem for BindingsGenerator as it only needs the tools project editor assembly.
+	// Generate the bindings here, before loading assemblies. The Godot assemblies
+	// may be missing if the glue wasn't generated yet in order to build them.
 	List<String> cmdline_args = OS::get_singleton()->get_cmdline_args();
 	BindingsGenerator::handle_cmdline_args(cmdline_args);
 #endif
 
-#ifndef MONO_GLUE_ENABLED
-	print_line("Run this binary with '--generate-mono-glue path/to/modules/mono/glue'");
-#endif
+	gdmono = memnew(GDMono);
+	gdmono->initialize();
 
 	if (gdmono->is_runtime_initialized()) {
 		gdmono->initialize_load_assemblies();
@@ -596,7 +598,9 @@ Vector<ScriptLanguage::StackInfo> CSharpLanguage::debug_get_current_stack_info()
 		return Vector<StackInfo>();
 	}
 	_recursion_flag_ = true;
-	SCOPE_EXIT { _recursion_flag_ = false; };
+	SCOPE_EXIT {
+		_recursion_flag_ = false;
+	};
 
 	GD_MONO_SCOPE_THREAD_ATTACH;
 
@@ -628,7 +632,9 @@ Vector<ScriptLanguage::StackInfo> CSharpLanguage::stack_trace_get_info(MonoObjec
 		return Vector<StackInfo>();
 	}
 	_recursion_flag_ = true;
-	SCOPE_EXIT { _recursion_flag_ = false; };
+	SCOPE_EXIT {
+		_recursion_flag_ = false;
+	};
 
 	GD_MONO_SCOPE_THREAD_ATTACH;
 
@@ -824,13 +830,13 @@ void CSharpLanguage::reload_assemblies(bool p_soft_reload) {
 		for (SelfList<ManagedCallable> *elem = ManagedCallable::instances.first(); elem; elem = elem->next()) {
 			ManagedCallable *managed_callable = elem->self();
 
-			MonoDelegate *delegate = (MonoDelegate *)managed_callable->delegate_handle.get_target();
-
 			Array serialized_data;
 			MonoObject *managed_serialized_data = GDMonoMarshal::variant_to_mono_object(serialized_data);
 
 			MonoException *exc = nullptr;
-			bool success = (bool)CACHED_METHOD_THUNK(DelegateUtils, TrySerializeDelegate).invoke(delegate, managed_serialized_data, &exc);
+			bool success = (bool)CACHED_METHOD_THUNK(DelegateUtils, TrySerializeDelegateWithGCHandle)
+								   .invoke(managed_callable->delegate_handle,
+										   managed_serialized_data, &exc);
 
 			if (exc) {
 				GDMonoUtils::debug_print_unhandled_exception(exc);
@@ -1149,10 +1155,11 @@ void CSharpLanguage::reload_assemblies(bool p_soft_reload) {
 			const Array &serialized_data = elem.value;
 
 			MonoObject *managed_serialized_data = GDMonoMarshal::variant_to_mono_object(serialized_data);
-			MonoDelegate *delegate = nullptr;
+			void *delegate = nullptr;
 
 			MonoException *exc = nullptr;
-			bool success = (bool)CACHED_METHOD_THUNK(DelegateUtils, TryDeserializeDelegate).invoke(managed_serialized_data, &delegate, &exc);
+			bool success = (bool)CACHED_METHOD_THUNK(DelegateUtils, TryDeserializeDelegateWithGCHandle)
+								   .invoke(managed_serialized_data, &delegate, &exc);
 
 			if (exc) {
 				GDMonoUtils::debug_print_unhandled_exception(exc);
@@ -1161,7 +1168,7 @@ void CSharpLanguage::reload_assemblies(bool p_soft_reload) {
 
 			if (success) {
 				ERR_CONTINUE(delegate == nullptr);
-				managed_callable->set_delegate(delegate);
+				managed_callable->delegate_handle = delegate;
 			} else if (OS::get_singleton()->is_stdout_verbose()) {
 				OS::get_singleton()->print("Failed to deserialize delegate\n");
 			}
@@ -1289,7 +1296,7 @@ bool CSharpLanguage::debug_break(const String &p_error, bool p_allow_continue) {
 	}
 }
 
-void CSharpLanguage::_on_scripts_domain_unloaded() {
+void CSharpLanguage::_on_scripts_domain_about_to_unload() {
 	for (KeyValue<Object *, CSharpScriptBinding> &E : script_bindings) {
 		CSharpScriptBinding &script_binding = E.value;
 		script_binding.gchandle.release();
@@ -1302,8 +1309,7 @@ void CSharpLanguage::_on_scripts_domain_unloaded() {
 
 		for (SelfList<ManagedCallable> *elem = ManagedCallable::instances.first(); elem; elem = elem->next()) {
 			ManagedCallable *managed_callable = elem->self();
-			managed_callable->delegate_handle.release();
-			managed_callable->delegate_invoke = nullptr;
+			managed_callable->release_delegate_handle();
 		}
 	}
 #endif
@@ -1455,7 +1461,7 @@ void CSharpLanguage::_instance_binding_free_callback(void *, void *, void *p_bin
 
 	if (GDMono::get_singleton() == nullptr) {
 #ifdef DEBUG_ENABLED
-		CRASH_COND(!csharp_lang->script_bindings.is_empty());
+		CRASH_COND(csharp_lang && !csharp_lang->script_bindings.is_empty());
 #endif
 		// Mono runtime finalized, all the gchandle bindings were already released
 		return;
@@ -1784,7 +1790,8 @@ void CSharpInstance::get_event_signals_state_for_reloading(List<Pair<StringName,
 		MonoObject *managed_serialized_data = GDMonoMarshal::variant_to_mono_object(serialized_data);
 
 		MonoException *exc = nullptr;
-		bool success = (bool)CACHED_METHOD_THUNK(DelegateUtils, TrySerializeDelegate).invoke(delegate_field_value, managed_serialized_data, &exc);
+		bool success = (bool)CACHED_METHOD_THUNK(DelegateUtils, TrySerializeDelegate)
+							   .invoke(delegate_field_value, managed_serialized_data, &exc);
 
 		if (exc) {
 			GDMonoUtils::debug_print_unhandled_exception(exc);