Frequently asked questions 2017-05-09T10:28:09+00:00

FREQUENTLY ASKED QUESTIONS

This is a short list of our most frequently asked questions. If you can’t find your question below, contact our support team and we’ll help you out!

Moving to Starcounter

With Starcounter you get performance in abundance, you can speed up your system, increase volumes in form of users, transactions etc. You can do things that were impossible before, reinvent your business and beat your rivals.

With Starcounter you just have to develop logic and User Interface, no more glue code moving data to and from the applications. Your cost for development and infrastructure will go down drastically. You can choose to use the time saved to develop much more functionality, giving you a serious competitive edge.

It will take you 2-4 weeks learning and trying before you get a hang of it. Spend the first week going through our tutorials, the second week with our sample apps, gradually shifting to building your own apps. We support you the whole way. Put your questions and thoughts on GitHub and we will reply promptly.

Starcounter as of today requires a Windows server. The server-side logic is written in Visual Studio using C#, SQL queries and REST/JSON. Future releases are planned to support Linux and Node.js. On the client-side we recommend a thin approach using Web Components with a minimum amount of Javascript. You are free to use other frameworks like AngularJS or ReactJS.

You can quickly replace your current database with Starcounter. The next step would be to make Wrapper apps around your existing applications and develop new apps in Starcounter, then gradually re-writing your old applications to Starcounter apps.

Pricing

Starcounter is free for developers and non-commercial use.
For commercial use and below 32 GB database the price is 10€/16GB/month. For commercial use and more than 32 GB database the price is 100€/16GB/month.
For ISV’s we propose individual pricing based on revenue sharing.
For support with SLA the price is 500€ per month/installation. Support without SLA is free.
To find out more, check out or pricing page.
Starcounter offer on-premise, cloud (public and managed), and hybrid deployments.
Unfortunately, today’s cloud hosting is expensive in respect to main memory costs. For instance, memory cost is a big price differentiator in Amazon EC2. Hence, today’s cloud providers aren’t suitable for large-loaded in-memory processing. However, it might turn out that the database size of your interest doesn’t demand big memory spaces.

For instance, today the largest production database we encountered is 80 Gb in size (when is operational in memory). More typical sizes for the largest customers are 30 to 50 Gb. The big tail of smaller customers run 4-16 Gb databases.

If one wants to run several customers with their own in-memory databases each and all on a single machine, a dedicated server hosting is a good choice. Today you can choose a dedicated machine with 64 Gb RAM on-board for 39 euro (https://www.hetzner.de/it/hosting/produkte_rootserver/ex41sssd). Even better choice is a co-location for a machine you assemble on purpose (with large RAM, which is relatively cheap if you choose a configuration yourself and purchase it).

Currently there is no implicit compression applied in Starcounter. The overhead seen between the raw data (e.g in text file form stored steel on disk) and operational data in the database in-memory is typical for relational databases (MySQL, SQL Server and alike) and is 2-5 times. Consider however that there is no redundant data instances in the application (since the database data and application data is physically the same), and for the performance reasons you will stop doing pre-aggregations, hence you will notice a decrease in memory consumption once adopting and running your solution in Starcounter.

About Starcounter

Starcounter is an application development platform for business applications. By bundling an in-memory database with a web application framework, it allows developers to quickly build a cooperating suite of fast micro-applications. Effectively creating business systems with high performance and great flexibility, while also cutting time to market.

In the core of Starcounter is a cognitive mapping engine that lets you combine yours and others’ micro applications into large business systems without the need to write integration code. The combination of our in-memory platform and the cognitive mapping creates what we call the world’s first Information Operating System.

Short answer: Yes.

Longer answer: Memory footprint goes down substantially with an in-memory processing solution. This is because there is no need to de-normalize data for performance. As a result, memory space requirement goes down. When it comes to live/operational data, the amount shouldn’t go beyond tens of gigabytes normally. The explanation is given here
“Let us analyze one of the largest commercial enterprises, Amazon, which has a yearly revenue of about 15 billion Euros. Assuming that an individual order line values at about 15 Euros and each order line incurs stored data of about 54 bytes – as specified for the TPC-C-benchmark –, we derive a total data volume of 54 GB per year for the order lines which is the dominating repository in such a sales application. This estimate neither includes the other data (customer and product data) which increases the volume nor the possibility to compress the data to decrease the volume. Nevertheless it is safe to assume that the yearly sales data can be fit into main memory of a large scale server”

If one is looking for a drop-in choice to do BI on Terabytes of historical data, then Cassandra, MongoDB aggregations or MapReduce-based solutions is a good choice. Starcounter is an excellent choice when you need to operate on (write and analyze) your live data in a mixed (read/write) workload with strong ACID transactions and high performance guarantees (millions of r/w transactions per second). The data is in-memory, which allows for real time BI. At the same time, performance provided on a single machine is enough to scale for more simultaneous users and increased application complexity from day one. An impact for the end user is excellent system responsiveness any time, more transactions processed, and analysis results delivered faster.

Based on this, we expect that your memory allocation requirements should be about gigabytes of data. If you have a mixed workload (say, you do BI together with other operations), Starcounter is a good choice. Modern apps are all about mixing apples and oranges: here you update data fields, in other parallel requests you calculate some stats from live data, at the same time as you get a request which traverses a graph of your logistic schemes (read) and updates certain vertices (write); so that the workload is truly mixed. Traditional multi-tier solutions usually fall short in those scenarios.

Starcounter has exceptional speed and handles real time transactions. This gives the capability to analyze data from large amounts of sensors and only save the results. With such a tool, you can handle most scenarios for IOT/IOE.
Very secure. Our database is fully ACID-compliant. ACID (Atomicity, Consistency, Isolation, Durability), means that the database transactions are processed reliably. Starcounter operates and stores all data in RAM and secures data and logs of the database on disk for fast recovery. For additional security Starcounter offers replication so you can have two servers with hot fail-over in case of hardware failure.

Our recommendation is to build your system on Starcounter without client-side logic, with only a thin client proposing changes to the server. This makes the system more secure from intrusion from the client side.

With Starcounter´s fine grained semantic object model, mirroring information to all other object models can be done on the lowest level with no data conversion. This makes integrations flexible and super easy. We call such applications “Wrapper apps”.
In Starcounter a data integration feature enables development of independent modules/ apps without the need for app-to-app integration or universal bus and master data management orchestration. Instead, data and functionality are integrated organically (and with minimal amounts of code) on the level of the in- memory platform and user interface, allowing for their efficient collaboration.
Since the applications Server and the database are totally integrated in Starcounter you do not need to write any code moving data to and from the database. There is no need for APIs, ORMs (Nhibertate, Entity Framework), microservices, etc. With Starcounter you only need to develop the logic on the server and the presentation on the client. This saves a lot of time. The move from a first prototype to a large production system does not requirement any refactoring.
We talk of reducing a need in hardware items, software licenses, administrative costs and electricity times 100x. Cases typically seen is (for retail business) a move from a cluster of Oracle appliances or a SQL Server federation (10+ or even 100 machines) to a single commodity server running Starcounter (plus one failover machine for a case of physical death of a server) with having equal or much better performance.

In return, customers have:
1) Substantially less TCO (total cost of ownership),
2) Solution simpler to use and maintain => faster time-to-market,
3) Greener solution (less electricity and emitted heat).

Once you as a customer experience problems with development agility, elasticity/scalability in number of users simultaneously connected/transactions per day, code complexity which impedes time to market, you have three choices:

1. Scale out the solution: rewrite your solution to be distributed, build a compute cluster, administer and procure the cluster, hire distributed systems developers, sacrifise data consistency or struggle maintaining it by doing ad-hoc transactional logic;

2. Scale up the solution: buy bigger/faster machine or (when this usually doesn’t help much) go for special hardware appliance 
from Oracle, SAP or IBM (priced can be up to 1’000’000$) and rewrite your software to get locked to these vendors, 
purchase licenses for production use (label starts with 100’000$);

3. Scale in the solution: keep your existing hardware (and sell unnecessary hardware) and move your solution to Starcounter, 
start to utilise your resource (hardware, people, power, time) with the extreme efficiency by letting Starcounter enable maximum out of a single commodity computer.

The in-memory processing gives additional benefits in terms of storage utilisation. For instance, in slow disk-based databases you do multi-dimensional pre-aggregations to aggregate results of analytical queries, or even run analytical processing over night in a separate data warehouse. With in-memory processing your current annual data is at hand just at a current moment to be crunched right when you cater for your daily data operations. Both OLTP and OLAP loads become possible simultaneously without redundant stores. You no longer need to overcome a performance bottleneck of disk-based database, instead you simply do processing of live data without impeding customers which use the system at a moment. That is called operational analytics. In-memory app platforms (such as Starcounter or SAP HANA) opens a way into it. Not only you shrink the storage space overheads, but you can do more in near-real time (for instance, make recommendations for your customers based on their current behavior, not on yesterday’s behaviour).

Today in Starcounter the default memory layout for a newly created database is 256 Mb minimum (the pages are preallocated but empty and to be filled with data). This is a tradeoff between pre-allocated memory size and actual size of data stored in a database, and works best for “one database – one machine – one customer” scenario, when the data tend to eventually grow out of 100+ Mb. For multi-tenant setups (once machine is occupied for many databases) this cap can be adjusted to lower sizes, depending on how much data a single customer will have at the lowest the optimal memory layout is calculated, so that multiple databases will live on a single machine friendly and compactly.
Starcounter is an in-memory application server which runs an in-memory relational database under the hood. Developers specify their schemas in C# code via classes (aka “code-first” in ORMs), and access data via Starcounter SQL, navigational properties and .NET in-memory LINQ. When the application is to run in Starcounter platform, Starcounter weaves the application, so that for the [Database] objects the .NET heap are replaced with Starcounter’s transactional heap. In this video you can see a (remarkably small) difference between the simplest .NET app with no database and the same app with Starcounter (full ACID database): https://www.youtube.com/watch?v=Xp7HVA5CPjQ (2 mins to watch).

Important to note is that all Starcounter apps are run inside the database. In a conventional way of building apps, you might have an in-memory database and an application (having its data objects in managed memory heap), and not only the database and application will have redundant data copies (row in a DB and a data object in the app’s heap), but also the chatter will appear to happen between the application and the database (via API calls or/and ODBC/JDBC drivers). It is known that this operational chatter (which also entails unnecessary data serialisation/deserealisation) is at least 30% and up to 90% of overall computing time in data processing applications. Hence, modern in-memory application platforms (like Starcounter or SAP HANA) remove the boundary between the business logic and the database and executes the whole business logic directly on a database memory. This eliminates 100x-1000x performance cap seen with conventional solutions. As a result, the whole application you write virtually becomes as a big stored procedure loaded into the database.

To be able to run your C# code inside a database, Starcounter gently hooks into MSIL when application is compiled by a .NET compiler. Thus, any 3-rd party library and code for .NET is interoperable with Starcounter, and you can very simply use all those in Starcounter apps. This is in contrast to other known in-memory app platforms, which provide their proprietary language/ runtimes they support solely. For instance, this is Transact SQL code for Microsoft SQL Server 2014/2016 Hekaton in-memory option, or ABAP SQL for SAP HANA. With Starcounter you continue to use the language and platform you know and like, however receiving all the benefits of in-memory processing with the code and the data being together at the smallest possible extent.

Starcounter also ships an Administrator panel, where you can issue SQL queries against your databases. Generally speaking, Starcounter SQL covers the majority of SQL-92 with own additions, like path expressions (http://starcounter.io/guides/sql/path- expressions/).

Each database is a separate OS process which owns the data and the application(s) code. You deilver applications to the database. You can upload one or more apps in a compiled form into the database and run them there. You can start and stop apps from Starcounter CLI (star, staradmin tools) and from graphical Administrator interface.

Table’s structure is defined in .NET classes definitions. When you need to change a schema, you need to provide a migration code which will take the data from the old data model and populate the database with this data for a new data model.

Usually you do not fine-tune Starcounter settings in conf files at all. However you might alter parameters such as number of schedulers (more cores you have on a machine – more schedulers you would like to run to efficiently utilise them with tasks) or number of workers in the web server. You can also alter web server settings to do simple rewrites (however for more complex processing and HTTPS we recommend to add a front-end server, usually residing on a separate machine).

Every Starcounter application is essentially a web application. You can expose REST handlers from within Starcounter application to build a RESTful app (http://starcounter.io/guides/rest/), you also utilise an efficient client-server communication protocol with a technology called Puppets (more on it here: http://starcounter.io/guides/web/).

You can setup a backup routine for your database on an operational (live) database using Windows shadow copy service, more on it here: http://starcounter.io/guides/tools/run-starcounter-in-production/.

You can run multiple independent applications as modules (sharing only a data model) together in a shared storage space, so that there is virtually no overhead of having these apps run alltogether and communicating in between. This gives a way for our customers to build highly modular customisable solutions.

When you run an ORM against a database, you get an overhead at least 10x even on simplest constructs. When the code gets more complex, you even start to remove ORM statements and write plain SQL (which you know the best how to form in the right way in respect to your app). Hence, Starcounter eliminates an ORM and instead replaces the application heap with Starcounter’s ACID database heap. The whole application becomese as one big stored procedure and the notion on ORM becomes wastly redundant and unnecessary. Starcounter developers continue using objects of .NET classes (marked as [Database] classes) just as if those objects are .NET objects, they use properties, functions and all other C# language facitilies.

This article explains the topic of Starcounter / ORMs / DDD (domain-driven design) in depth, we higly recommend you to get accustomed: https://github.com/miyconst/shopping-cart/wiki/Starcounter-vs-ORM-and-DDD

Other

Starcounter delivers high-volume transaction processing (OLTP) and high volume real-time analytical processes (OLAP) based on a unified data model without the redundant data layers typically required by traditional RDBMS based systems. Examples for redundant data layers are custom-built layers based on database tuning efforts such as secondary indexes, or application built-in performance accelerators such as aggregate tables or multiple general ledger versions for different managerial reporting needs.

The massive simplifications of the data model and the data processing layers enable business and technological innovations on a broad scale across all lines-of-business and industry solutions. The new application architecture simplifies system landscape architectures and accelerates cloud deployments.

There is no other combined in-memory application server and database. The closest is SAP Hana but it is not fully combined and works only in the SAP world.
1. Developers keep focus on their subject domain and business value. They do not spend time a) removing bottlenecks in the ORM and tuning SQL queries, b) architecturing complicated scale-out clustered solutions, c) building polyglot persistance architecture (e.g. running two or more DB vendors – one document, one graph-based for performance sake and/or development availability), d) wrapping abstractions on abstractions to isolate raw data management from business objects, e) adding 2-3 caching layers around the DB/caches, and so on. Instead, they write clean and consize code with no glue code (the code which ties together the database, the applications layers, the web communication, the tasks scheduling and so on). In result they have a very elastic solution which sustains 10’000+ simultaneously operating users on a single machine. In return, for our customers this increases time to market and competitive edge.

2. Freedom to operate in code. In Starcounter, where your data model is transactional, persistent and high-performant in- memory, you are untied to build a model of virtually any complexity. Be it a graph-based model, document-based model, purely relational model or any mix of those, you can afford to have it (in terms of data size and performance). Having this model in place, you do not need to think of questions like “We cannot have 2, 3 and more JOINS in that query, since it will not perform at all”. In Starcounter, JOIN is virtually a derefencing of an in-memory pointer, which is superfast. The differences we’ve seen in simplest applications comparing Starcounter to conventional DBs (with or without an ORM layer in the code) varies times 100x-8000x. When developers switch to Starcounter, they generally learn to stop avoiding JOINs and other unnecessary database-related ramifications and restrictions.

3. Highly modular way to develop applications. Starcounter platform enables hosting multiple independent applications sharing a single data model to constitute your business solution in a highly composable way. More on this in below sections.

Today Starcounter provides an async replication facility as part of distribution. You can setup replication for an arbitrary tree of Starcounter instances. This facility can be used:

A) as a classical full-data master-slave,
B) as a partial master-master (for disjoint sets of data when one part of the database tables is master data to the replica but the other part is a mirror of the data that is being changed on a replica) or partial master-slave, with arbitrary filtering on data entries,
C) as a master-master active-passive (standby instance for failover). You can also put a load balancer in front of Starcounter (usually a separate machine, HAProxy or nginx) to track failover events.

Please note that you unlikely will use replication to increase (scale-out) read performance. Starcounter is good enough to provide a desirable performance on a single instance (http://starcounter.com/resources-activities-articles-and-knowledge/starcounter- performance/).

The two major use cases for replication are:

1) Support failover scenario;
2) Support high data availability and low latency, when you want to have a read/write replica at hand, e.g.:
2.1) when point of sales operates disconnected from the mother machine (e.g. store is located in the mountains and the connection quality is poor or connection is missing);
2.2) when a vessel travels in the seas, connects to the ground machine via satellite with disturbances and disconnections;
2.3) Geo-distributed applications, when you want to have replicas of data physically close to users to decrease the connection latency (e.g. run a dedicated replica in China for Chinese users, while having the core business and its servers in Germany).

In the replication facility Starcounter utilises a module called log streamer, which resolves stable replication even over poor network having connection drop-offs and other quality of service degradations. The log streamer aims to efficiently utilise the connection channel and sync data between two parities appropriately on the connection being re-established.

Note that simply letting an async replica take over the failed master isn’t generally enough for failover, since some data might have been reported committed on the master not having reached the replica (due to network loss, master machine network card dead and so on). However, providing a synchronous replication (when the replica acknowledge master to have the data received and master reports transaction committed based on this replica’s feedback) is not a practical choice: master becomes dependant on replica, and the overall throughput of the master system is now bound to the network and to the death of replica. Hence, the active-passive master-passive setup (when one machine is a master, but when the master dies the async replica becomes the master) is a practical choise today (not only for Starcounter installations).

However, a question for small data portions being potentially lost still prevails. In order to support full-consistent failover without sacrificing performance (by sync replication), we will support the following failover setup in Starcounter in one of the future version updates.

1) On a master machine transactional log is written to the fault-tolerant storage (like RAID-1 with multiplexing). When master dies, we expect the fault-tolerant storage to tolerate the fault and redundantly preserve the data. Lower write performance of fault- tolerant stores isn’t an issue for Starcounter, since it is an in-memory database technology, hence it persists data to disk in a sequential manner (rather than writing to arbitrary locations on a disk), enabling to stream gigabytes of writes per seconds to persistent stores.

2) On master death the load balancer / watchdog (usually run on a separate machine close to master machine) tracks a need to pass processing to the standby replica machine (by loosing heartbeat from the master or alike).

3) Standby replica machine takes over. Replica already has all or almost all data from the master. To ensure all data is presented (and fetch the small missing bits if needed) the replica machine reads up from the fault-tolerant storage of the master and syncs replica’s data. Since the diff between master and replica is usually no more than seconds (either because of network data loss, or for the big distance between two machines), it will take a fraction of second or seconds to read up the needed bits (since there is no need to load the whole database into memory from disk, which might take 3-5 minutes for 30-50 Gb database).

4) After the sync is completed, replica becomes a master available for the users.

By continuing to use the site, you agree to the use of cookies. More information

The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.

Close