Canonical Ledgers

David Chapman

Factomize
Thank you for the application! A quick question to start:

A.
According to our calculations, if we run two authority servers at 50% efficiency with two full time salaries we roughly break even at a factoid price of $11.30.
Are you willing to share this calculation with us?
 
BTW, I put that we were on the testnet since February 2, which was my hazy recollection of when it started. Upon discussion with others it appears the testnet may not have started until a week or so later. I wasn't trying to invent extra time on the testnet I just have a poor long term memory. We were on the testnet since the Alpha stage whenever in February that began. :)
 
Last edited:

David Chapman

Factomize
Thank you. $4,000 is not a full time salary for the resumes you guys bring to the table but that explains where the numbers came from. I personally hope that you guys will be able to draw a more realistic salary in the future if you are elected to run an Authority Node. In my opinion, everyone who runs an Authority Node should earn a fair salary. I do understand why you're keeping it low, for now, and appreciate the clarity.

At some point this thread will be made public. Is it ok that the Community sees that spreadsheet?
 
Last edited:
Adam and I agree that $4000 is not a fair salary for what we could make in industry, but we're willing to go that low (and lower honestly) for a period of time in order to survive deep bear markets. Instead of raising external capital and giving up some control of the company and direction, we both have personal savings we can use to buffer bad times. Given a few months at current Factoid prices we'll be able to build up company reserves which will help mitigate market down-turns. Additionally, we plan to scale the salary up as we can afford to do so, we just tend to be extra-conservative because we want to create a sustainable company to be in this for the long haul.

We're fine with making the spreadsheet public, but I can always change the permission back to private if we need to.
 

Niels Klomp

BI Foundation
Core Committee
Governance Working Group
Thank you for your application

NK1)
You mentioned 2 nodes for the testnet in the form. Could you point out for what purposes you will use these nodes?
 

Niels Klomp

BI Foundation
Core Committee
Governance Working Group
NK2)
We could tolerate prices as low as $7.50 for some time. We believe in Factom and its potential and we are willing to risk running at break even or a small loss for up to 6 months.
Could you quantify "a small loss"?
 

Niels Klomp

BI Foundation
Core Committee
Governance Working Group
NK3)
We are aware that Factom Inc offers a similar service and they have told us that they would not view this as competition. We plan to launch this service on entrycredits.io later this year.
I like that you contacted them about it. But what would you do if they told otherwise?
 

Niels Klomp

BI Foundation
Core Committee
Governance Working Group
NK4)
We believe Factom has the potential to be the global data layer for the Internet. A single, decentralized, global source of truth and facts.
Could you elaborate a little more on these claims?
 

Niels Klomp

BI Foundation
Core Committee
Governance Working Group
NK5)
Upstream updates to software are available on Arch Linux much sooner than on most other Linux distros.
Could you elaborate on this claim? Could you include possible positive and negatives in your answer? Could you point out the key difference with a distribution like Ubuntu or Debian regarding your claim?
 

Niels Klomp

BI Foundation
Core Committee
Governance Working Group
NK6)
You mention a bastion server with 2FA. Could you disclose how this server is secured? Could having a bastion server also be problematic in some cases? If yes, how would you resolve?
 

Niels Klomp

BI Foundation
Core Committee
Governance Working Group
NK7)
You financials seem to indicate that you both will be working full time for Canonical Ledgers. Is it correct that you would give up your current jobs?
 

Niels Klomp

BI Foundation
Core Committee
Governance Working Group
NK8)
You mentioned that all SSH has been shielded and only allowed from the bastion server. Is this reflected in your architecture picture? If not could you elaborate?
 
Thank you for your application

NK1)
You mentioned 2 nodes for the testnet in the form. Could you point out for what purposes you will use these nodes?
Note: This is Adam Levy responding.

Participating in the testnet serves a few purposes. First, it keeps us up to date with what is happening in the development of the Factom protocol and with changes to the software that we may need to plan for going forward. Additionally, this helps us stay in direct touch with the Factom developers. We want to help find issues and offer suggestions or pull requests to help move the software and protocol forward. Second, it gives us a place to test any changes to our own infrastructure or tools that we are developing.

Having two nodes allows us to properly perform updates and brainswaps. We expect we may set up and tear down nodes from time to time as we experiment and play with our infrastructure on the testnet. So we just budgeted for 2 reasonably sized ec2 instances.

At the moment we have 3 nodes running on the testnet. Two have identities and we use them for brainswaps. The third is a follower node used for generating test loads and experimenting with the testnet.
 
Last edited:
Sorry for the delay. Here are the rest of our responses to your questions.

NK2)

Could you quantify "a small loss"?

The amount of loss the company can tolerate is dependent on how much reserves it has built up. If we get current or better Factom prices for a few months we’ll be able to build up some cash reserves to get us through a bear market. Without any cash reserves our plan is to first scale founders salaries down to our absolute minimum ($2k each) and then to reduce server costs by downgrading server specs, to keep the company in business. We both have personal savings that we can use to cover the shortfall between our minimum salaries and our actual expenses and we are willing to burn that for up to six months. Here’s our Google financial spreadsheet with a FCT price of $7.50 plugged in and with our salaries scaled down to our minimum. https://docs.google.com/spreadsheets/d/1wVJgUWKWZsk94aH69arXkLyzg3zMy4lEDzY-uWqI_do/edit#gid=0


You can see the company will still have a tiny profit at that rate which we would use as a buffer for unforeseen expenses.
 
NK3)

I like that you contacted them about it. But what would you do if they told otherwise?
Entry credits are the core utility token required to use Factom, so it would have caused some questions in our minds about Factom's core goals if they had told us that selling entry credits was something they wanted exclusive control over. At minimum, it would have given us pause regarding mentioning it in our application. Whether that would have totally deterred us from pursuing it as a potential revenue stream depends a lot on how strongly Factom felt against us providing that service. We would have to weigh the potential social costs against the potential benefits. But given that it is not currently a highly demanded service it probably wouldn't have taken a whole lot to deter us if that was truly frowned upon in the community.


The main reason we approached Paul Snow about it was more to find out if there were any pitfalls or issues that he might be willing to share about running such a service. We also of course didn't want to risk stepping on Factom Inc's toes but we would have been a bit surprised had they taken issue with it.
 
NK4)

Could you elaborate a little more on these claims?
It seems safe to predict that in 5-10 years there will still be numerous blockchains in different niches, and that a handful of winners with large network effects will dominate the ecosystem. As we’ve already seen, technology developed in this space is quickly democratized. Large network effects are what ultimately will drive the winners. Much like social platforms, public blockchains gain utility through their mass adoption.

Since down the road the Factom protocol could secure data on any number of blockchains, we believe it will attract serious industry players who are less inclined to trust their data integrity to a more volatile blockchain. Additionally, Factom’s priority on data integrity keeps the protocol simple and narrowly focused. We believe this will prove to be a strength within an otherwise complex ecosystem which frequently seems to assume that all data and business logic should reside on the blockchain. The truth of the matter is that centralized systems are, and will continue to be, more efficient than decentralized solutions. However, the core issue with centralized systems continues to be a lack of transparency. Factom allows a straightforward way to bootstrap the security of the blockchain into existing centralized business logic.

Moreover, Factom Inc’s strong focus on Factoids and Entry Credits as utilities, is making it one of the few US based blockchain companies that is successfully navigating the regulatory environment. As the Factom community grows, we expect Factom will continue to stand out as a trusted protocol with a strong and focused core of users and developers.
 
NK5)

Could you elaborate on this claim? Could you include possible positive and negatives in your answer? Could you point out the key difference with a distribution like Ubuntu or Debian regarding your claim?
As a rolling release distribution, Arch Linux package maintainers test and release updates to software as they are published by the upstream developers. Accordingly, the Arch Linux distribution has no version number itself and most packages do not specify version numbers in their dependencies. Software is linked against whatever the latest libraries are at the time, so users are expected to upgrade all packages at once. Performing partial upgrades is a common cause of breakage among inexperienced users and is not supported. The effect is a very up to date distribution that gets frequent incremental updates. With only minimal user intervention, it is possible to take an old Arch Linux image going back years, install it, and bring it fully up to date.

Contrast this with Debian based distributions which have OS version numbers and version locked libraries. Debian based distributions offer support for some time for the set of software and libraries of each release. Software is linked against the libraries available for that version of the OS. This works well for distributions that generally ship desktop environments and cater to users who want it to “just work” without them necessarily having to take responsibility for updating or understanding their systems. One downside is that these distributions generally make some strong assumptions about how their users want software to be pre-configured or what should be pre-installed. Additionally, package maintainers sometimes must patch upstream software to work with older libraries. This approach requires a slower release cycle.

There are those who feel that rolling release has no place in production environments citing issues of stability and too frequent updates. However, my understanding and experience of running Arch Linux in a critical production environment, on local armv7 servers, in the cloud, and on my personal systems, runs counter to this. It is true that things change more often with rolling release, but when and how those updates are applied is still under the control of the user. It is also true that updating a rolling release distribution generally requires careful attention, but so should updating any production system. Firing off updates on a production server without a proper testing environment is dangerous regardless of the operating system you use.

The main advantages of Arch Linux for some people and circumstances, are also the main detractors for others. For experienced Linux users who have the time and desire to build up a minimalist system, Arch provides an excellent, stable, minimalist starting point with excellent documentation, a responsive community, and a straight forward set of packaging tools. We view the extra attention and care required to maintain a rolling release distribution to be both worth the benefits of up to date software, and largely consistent with what we would need to do for any other distribution anyway.
 
NK6)
You mention a bastion server with 2FA. Could you disclose how this server is secured? Could having a bastion server also be problematic in some cases? If yes, how would you resolve?
We have taken a number of measures to secure our bastion server. First and foremost we are using IP specific external firewall rules. This blocks all traffic not coming from our IP addresses preventing any potential DDoS attacks. Occasionally this is inconvenient if using a mobile hotspot and in such cases we will open up this firewall rule. But we also run fail2ban, which drops all login attempts from IP addresses that have failed to login more than 3 times in a row.

Second we are using 2FA and non-standard user accounts. The first factor is password encrypted RSA 4096 keys stored on our local laptops and the second factor will be Google Authenticator on our phones. We both take security on our laptops and phones very seriously and we will be re-evaluating and revising our security models if we get elected.

Our AWS account is secured with properly configured user accounts, strong passwords, and Google Authenticator. We use an encrypted password manager with a strong master password and yubikey OTP 2FA.

Finally our bastion server is subject to the same monitoring as all of our servers. If a memory leak or an abnormal number of network connections occurred we would be notified promptly and have time to get ahead of the situation.

In a worst case scenario, where the bastion server went down, we would temporarily be unable to login to our authority servers. The simplest solution in this scenario is to launch a new ec2 instance from an existing snapshot of the bastion server and assign it an IP address on our private subnet that is whitelisted by the authority servers. This would take maybe a half hour at most and is an incredibly unlikely scenario as the bastion is well secured and is not running any extraneous software.
 
NK7)
You financials seem to indicate that you both will be working full time for Canonical Ledgers. Is it correct that you would give up your current jobs?
Adam is currently between jobs and is working full time on Canonical Ledgers. If elected, Adam will continue working full-time for Canonical Ledgers. Sam will transition to full time over the next couple of months as he wraps up his current work obligations. We both want to be working full time on Canonical Ledgers as we believe there are many opportunities in the Factom ecosystem to generate value both for us and the wider community.
 
NK8)
You mentioned that all SSH has been shielded and only allowed from the bastion server. Is this reflected in your architecture picture? If not could you elaborate?
It is depicted in the architecture diagram by the rectangle around the authority servers labeled “VPC Private Subnet and Firewall” and the arrows between the various servers. Some elaboration is worthwhile, though. The authority servers will actually not even have a public IP address at all. So they will only be accessible within our private subnet on AWS. In order to access one of the authority servers, you first have to SSH to the bastion server and forward your keys. From there, you can SSH anywhere within the private subnet. Additional internal firewall rules will ensure that all servers will only accept SSH connections from the bastion host.

One thing that is not clearly shown in the diagram is how Factom Inc will connect to the authority server’s factomd_node docker container. In the diagram it shows them connecting directly through a hole in the private subnet and firewall. From their perspective this is what is happening. But in reality, they will be connecting through one of the guard node servers. We will use SSH port forwarding to forward the docker container’s SSH port to the same port on the guard node. This allows us to seamlessly switch between the two private nodes when we perform a brainswap.
 

Matt Osborne

Go Immutable
Exchange Working Group
Legal Working Group
Hi Adam and Samuel,
You guys have one of the smallest teams in the application pool. As Factom grows, the competition to run nodes is only going to increase (and probably exponentially). What are your plans to scale? What niche do you see yourselves filling? Thanks
 
Hi Adam and Samuel,
You guys have one of the smallest teams in the application pool. As Factom grows, the competition to run nodes is only going to increase (and probably exponentially). What are your plans to scale? What niche do you see yourselves filling? Thanks
We’re glad to have a chance to address this. While some might look at having a small team as a weakness, we feel that keeping our business lean is an advantage at this early stage. There is no doubt in our minds that the two of us can support running our authority nodes with high reliability and security just as well as a larger team, and we can continue to afford to do so through severe bear markets.

However, as the price of FCT supports a larger team we plan to hire additional people to meet the demand for on-call support and to help code up the projects we will be working on. We envision our company growing with the network both as the price of FCT supports it and as we bring in additional revenue from paid private projects. We believe that it is prudent to run a lean team until the protocol is proved out and the ecosystem grows. This way we keep our FCT liquidation rate fairly low, allowing us to hold and stake more.

In the coming months we will continue our involvement with the testnet, providing community support and contributing to tools and projects like the digital identity project that we have agreed to collaborate on with Factomatic. Sam is currently one of the interim testnet coordinators. Adam is also working on an alternative approach to monitoring factomd nodes that will allow for secure central monitoring of all nodes. Additionally, any tools or techniques that we develop internally for our own stack will be open sourced.

Aside from community projects, we plan to run a lean startup design thinking process to identify key high value Factom integrations. If that sounds vague it’s because at this point it is. We have several ideas that we are still researching revolving around brand and art authenticity and media source authenticity. Both of these will likely depend on the work we plan to do on Factom digital identities.
 

David Chapman

Factomize
B. Let's say you get an Authority Node. How it performs will be easy to monitor. However, as part of your campaign you also discuss additional development work which is important since your single node Efficiency is set at 25%. The Standing Parties will likely want to see what progress you're making since it was part of your campaign. My questions are:

1. How would you communicate with the Standing Parties. Examples would be blog, twitter, Discord, Reddit, etc.
2. What would you communicate and how often?
3. What metrics should be used to gauge your success?
 
B. Let's say you get an Authority Node. How it performs will be easy to monitor. However, as part of your campaign you also discuss additional development work which is important since your single node Efficiency is set at 25%. The Standing Parties will likely want to see what progress you're making since it was part of your campaign. My questions are:

1. How would you communicate with the Standing Parties. Examples would be blog, twitter, Discord, Reddit, etc.
2. What would you communicate and how often?
3. What metrics should be used to gauge your success?
We will continue to actively participate on Discord, so we will be reachable there and our participation in the testnet will be visible. We also plan to have a web presence where we will post major milestones, quarterly updates, and write-ups detailing our infrastructure or any tools or techniques we develop for maintaining an Authority node.

We believe that our success should be judged primarily by our contributions to the community. In our opinion helping people and organizations get up to speed on Factom is hugely valuable. As is contributing to projects that build infrastructure or interest around Factom. All of our projects will have clearly defined objectives, milestones and other project tracking metrics. So our progress in these areas will be transparent.
 
Last edited:
Top