Apache Ignite

As the need for database scaling and in-memory data processing increases, so does the search for a powerful tool that can meet these needs. Apache Ignite is an open-source, distributed in-memory data platform that provides a solution to this need. In this article, we'll explore Apache Ignite, look at its core features, and see how to integrate it with .NET Core through a sample project.

What is Apache Ignite?

Apache Ignite is a scalable and high-performance data platform that uses in-memory technologies for data storage and processing. Ignite not only stores data in memory but also offers distributed data processing, SQL and NoSQL data management, data grid, and much more. These features enable you to develop low-latency and high-performance applications.

Ignite is a widely preferred tool for solving database scaling issues or intensive data processing needs. It's an ideal platform especially for big data and real-time processing requirements.

Core Features of Apache Ignite

• In-Memory Storage: Ignite provides high-speed access by storing your data in memory.
• Distributed SQL: You can access data using SQL queries, even in a distributed environment.
• Horizontal Scaling: Ignite can easily scale horizontally by adding nodes.
• Low Latency: Since Ignite keeps data directly in memory, it offers millisecond-level latency.

.NET Core Integration with Apache Ignite

Apache Ignite, with its .NET Core support, allows you to use in-memory data storage and distributed data processing capabilities in your .NET applications. Let's see how we can use Ignite with .NET Core.

Step 1: Adding the Ignite Library to Your Project

First, we'll start by installing the required NuGet package for Ignite's .NET Core support. You can install this package using the following command:

Install-Package Apache.Ignite

Step 2: Starting the Ignite Server

We'll create a simple console application to start the Ignite server. The Ignite server is the main component we'll use to store data and perform distributed processing. The following code snippet shows an example you can use to start the Ignite server:

using Apache.Ignite;
using Apache.Ignite.Core.Cache.Configuration;
using System;

namespace IgniteDotNetExample
{
    class Program
    {
        static void Main(string[] args)
        {
            var igniteConfiguration = new IgniteConfiguration
            {
                IgniteInstanceName = "myIgniteInstance",
                WorkDirectory = "./igniteWorkDir",
                ClientMode = false // Running as a server node.
            };

            IIgnite ignite = Ignition.Start(igniteConfiguration);

            Console.WriteLine("Ignite server started.");

            // Let's create a sample cache.
            var cacheConfiguration = new CacheConfiguration
            {
                Name = "sampleCache",
                CacheMode = CacheMode.Partitioned,
                AtomicityMode = CacheAtomicityMode.Transactional
            };

            var cache = ignite.GetOrCreateCache<int, string>(cacheConfiguration);

            // Let's add data to the cache.
            cache.Put(1, "Hello Ignite!");
            string value = cache.Get(1);

            Console.WriteLine($"Data in cache: {value}");

            Console.ReadLine();
        }
    }
}

In the code above, the Ignite server is started and a cache named sampleCache is created. This cache will be used to store our data in memory.

Step 3: Two Ignite Instances and Persistent Storage Structure

Let's prepare an example showing how Apache Ignite works with two instances and how to enable persistent storage. In this example, we'll start two Ignite nodes and configure persistent storage to save data on disk.

using Apache.Ignite.Core;
using Apache.Ignite.Core.Cache.Configuration;
using Apache.Ignite.Core.Configuration;
using System;

namespace IgnitePersistentExample
{
    class Program
    {
        static void Main(string[] args)
        {
            // Starting the first Ignite instance
            var igniteConfig1 = new IgniteConfiguration
            {
                IgniteInstanceName = "igniteInstance1",
                WorkDirectory = "./igniteWorkDir1",
                DataStorageConfiguration = new DataStorageConfiguration
                {
                    DefaultDataRegionConfiguration = new DataRegionConfiguration
                    {
                        Name = "Default_Region",
                        PersistenceEnabled = true // Persistent storage enabled.
                    }
                }
            };

            IIgnite ignite1 = Ignition.Start(igniteConfig1);
            ignite1.GetCluster().Active(true);
            Console.WriteLine("Ignite Instance 1 started.");

            // Starting the second Ignite instance
            var igniteConfig2 = new IgniteConfiguration
            {
                IgniteInstanceName = "igniteInstance2",
                WorkDirectory = "./igniteWorkDir2",
                DataStorageConfiguration = new DataStorageConfiguration
                {
                    DefaultDataRegionConfiguration = new DataRegionConfiguration
                    {
                        Name = "Default_Region",
                        PersistenceEnabled = true // Persistent storage enabled.
                    }
                }
            };

            IIgnite ignite2 = Ignition.Start(igniteConfig2);
            ignite2.GetCluster().Active(true);
            Console.WriteLine("Ignite Instance 2 started.");

            // Creating cache and adding data
            var cacheConfiguration = new CacheConfiguration
            {
                Name = "persistentCache",
                CacheMode = CacheMode.Partitioned,
                AtomicityMode = CacheAtomicityMode.Transactional
            };

            var cache = ignite1.GetOrCreateCache<int, string>(cacheConfiguration);
            cache.Put(1, "Persistent Hello Ignite!");
            string value = cache.Get(1);

            Console.WriteLine($"Data in Persistent Cache: {value}");

            Console.ReadLine();
        }
    }
}

In the code above, two different Ignite instances are started with persistent storage configuration enabled. This way, even if the Ignite servers are restarted, the data will be saved on disk and won't be lost.

Step 4: Accessing Data with Ignite

One of Ignite's most powerful features is SQL support. You can process data stored on Ignite using SQL queries. For example, it's possible to store a user table on Ignite and access the data in this table using SQL queries:

[Serializable]
public class User
{
    [QuerySqlField(IsIndexed = true)]
    public int Id { get; set; }

    [QuerySqlField]
    public string Name { get; set; }
}

class Program
{
    static void Main(string[] args)
    {
        IIgnite ignite = Ignition.Start();

        var cache = ignite.GetOrCreateCache<int, User>(new CacheConfiguration("userCache", typeof(User)));

        // Let's add users.
        cache.Put(1, new User { Id = 1, Name = "Ali" });
        cache.Put(2, new User { Id = 2, Name = "Ayse" });

        // Let's fetch users with an SQL query.
        var query = new SqlFieldsQuery("SELECT Id, Name FROM User WHERE Name = ?", "Ali");
        var cursor = cache.Query(query);

        foreach (var row in cursor)
        {
            Console.WriteLine($"User: Id={row[0]}, Name={row[1]}");
        }

        Console.ReadLine();
    }
}

In the example above, we created a class named User and stored it in a cache named userCache. Then, we accessed the data stored in this cache using SQL queries.

Conclusion

Apache Ignite is a powerful platform that offers scalable and high-performance in-memory data management and processing solutions. By storing data in memory and providing distributed SQL support, Ignite can help accelerate your .NET applications and increase their efficiency. In this article, we covered the core features of Apache Ignite and how it can be integrated with .NET Core through examples. We also learned how to set up a structure working with persistent storage features using two Ignite instances.

To discover other powerful features that Ignite offers and learn more, you can check out the official Apache Ignite documentation.

Comments and Contributions

Feel free to comment to share your experiences or ask questions about Apache Ignite and .NET Core. For those who want to learn more about developing real-time data processing applications with Apache Ignite, Ignite's powerful features are truly worth exploring.