Matter of Fact, LLC

NK02)
Node #1: Connection & uplink speed (not just your NIC speed) *
Please take the lowest value of your Network Interface Card and any intermediary until the last gateway to the internet from your datacenter
You replied with 2.7Tbps. Are you sure this is the correct answer to the above question? If not, could you give the correct number
 
General question:
This is arguable the second or third time you deviate from the answers provided;
Could you explain why you did not answer the question; it clearly stated the purpose; It was a question with multiple answers (checkboxes), including "other", where you answer in a completely different direction then all available answers.


NK05)
How does your team administer the nodes (more options possible)? *
Anders:

Rico and Perly use key-based auth and ip whitelisting to manage via ssh. Everyone else will use the Proxmox [or later bespoke PWA] web UI combined with Cloudflare 2-factor authentication.
You later state that 4 people are able to operate the servers. Since you did not answer the intent of the above question. Could you list your direct team members that would be able to do this? Could you break down the years of experience running production machines per team member?
Do you use external administrators? If so are these administrators known (per administrator) beforehand?
 
NK07)
A simple PWA will be deployed to grant non-technical stakeholders emergency access to rudimentary service verbs [down, update, up, start]. Using a PWA will permit it to be installed into their phone home screens and receive notifications [which will auto-fire to non-tech staff past a set threshold].
Given the stability of Factom M3 and the coordination required between technical staff, do you find the above (including update) a viable solution? Please elaborate the reasoning
 
NK02)

You replied with 2.7Tbps. Are you sure this is the correct answer to the above question? If not, could you give the correct number
Was not clear if you wanted the data center total from the copy I was working from. The data link is gbps guaranteed, backed by the total above level of peering.
 
NK03)

Why did you choose MaxIOPS instead of SSD, you already mentioned the MaxIOPS?

NK04)

You replied with "yes"
Which one is it or maybe both? Please note that all options were listed in the checkboxes
Rico here. I didn't submit the questions or see the form you're talking about. I worked off a spreadsheet which clearly lacked some details. My apologies for being unclear.

We chose "maxiops" 'over' SSD because that's what upcloud calls their (assuredly SSD-based) storage layer.

As to volume or disk separation, we plan to use a separate volume.
 
Last edited:
General question:
This is arguable the second or third time you deviate from the answers provided;
Could you explain why you did not answer the question; it clearly stated the purpose; It was a question with multiple answers (checkboxes), including "other", where you answer in a completely different direction then all available answers.


NK05)

You later state that 4 people are able to operate the servers. Since you did not answer the intent of the above question. Could you list your direct team members that would be able to do this? Could you break down the years of experience running production machines per team member?
Do you use external administrators? If so are these administrators known (per administrator) beforehand?
Dan, the team leader will field this himself once I'm done with the technical questions.
The tldr is, he submitted the form after I provided information into a spreadsheet our entire team was collectively working on. Apologies for the confusion.
 
NK03)

Why did you choose MaxIOPS instead of SSD, you already mentioned the MaxIOPS?

NK04)

You replied with "yes"
Which one is it or maybe both? Please note that all options were listed in the checkboxes
NK07)

Given the stability of Factom M3 and the coordination required between technical staff, do you find the above (including update) a viable solution? Please elaborate the reasoning
This option will be a last ditch fail safe that will almost certainly never be used. This will not be the common administration pattern, primarily it will provide alert notifications. There will also be thorough documentation and an emergency hand off one sheet for such a situation, but not every situation would require this (I. E. Standard upgrade)
 
General question:
This is arguable the second or third time you deviate from the answers provided;
Could you explain why you did not answer the question; it clearly stated the purpose; It was a question with multiple answers (checkboxes), including "other", where you answer in a completely different direction then all available answers.


NK05)

You later state that 4 people are able to operate the servers. Since you did not answer the intent of the above question. Could you list your direct team members that would be able to do this? Could you break down the years of experience running production machines per team member?
Do you use external administrators? If so are these administrators known (per administrator) beforehand?
Dan here, I also apologize for any confusion. As the application process was defined and refined (daily), we worked off the best documents provided at the time.

In regards to the team members that will be able to operate the servers: Rico is the architect of the system with a combined 24 years experience running production machines. Perly is second in command with over 10 years experience.

As stated in our application, “A simple PWA will be deployed to grant non-technical stakeholders emergency access to rudimentary service verbs [down, update, up, start]. Using a PWA will permit it to be installed into their phone home screens and receive notifications [which will auto-fire to non-tech staff past a set threshold]”. "Alerting services set up to simultaneously notify (at least) two geographically disparate technical staff who rotate primary on-call responsibilities. We will thoughtfully engineer systems which permit N+1 failsafes in the unlikely event that multiple technical staff are unreachable”.

Brooke has 6 months experience in setting up and running a node for a different cryptocurrency. Rico and Perly will ensure both Brooke and Dan are trained as fail safe administrators. At this point, the use of external administrators is dependent on how the needs of our business expands and we are confident within the 34.5 yrs of combined experience and capabilities of this core team.
 
Last edited:
Rico here. I'd also like to say one other thing regarding the apparent professionalism of the form as submitted:

I'm delighted to see the professionalism Niels (and David) are bringing to this process. Some of the phrasing that I'm responsible for in the form was more flippant than it could have been. I am able to write more tersely and professionally but was keeping things more in line with the tone of much of Factom's public-facing documentation ;). I welcome the improvement in professionalism.

NK06)

Please elaborate
Not sure how far down the sysadmin best practice rabbit hole you would like me to go? We haven't finalized our specification but by ruthlessly employed I mean I intend to be on the absurd end of paranoid about protection. 2-Factor authentication on everything, IP whitelisting, no listening services, etc. Key handling will be incrementally secured levels, with Dan having ultimate access. Please let me know if there's more clarification on a specific angle you require. Thank you again :)
 
Thx for the compliment, appreciated.

Well you can take me down the rabbit hole pretty extensively. Have some knowledge about server systems ;)

What type(s) of firewall will you be using? What will you be securing?
Could you list some techniques on the OS level you will be taking to secure the server?

Just some additional information about the above is sufficient.
You will probably get that the Auth nodes and it's security play a vital role in providing the thrust in a decentralized system, that is not fully decentralized
 
Dan here, I also apologize for any confusion. As the application process was defined and refined (daily), we worked off the best documents provided at the time.

In regards to the team members that will be able to operate the servers: Rico is the architect of the system with a combined 24 years experience running production machines. Perly is second in command with over 10 years experience.

As stated in our application, “A simple PWA will be deployed to grant non-technical stakeholders emergency access to rudimentary service verbs [down, update, up, start]. Using a PWA will permit it to be installed into their phone home screens and receive notifications [which will auto-fire to non-tech staff past a set threshold]”. "Alerting services set up to simultaneously notify (at least) two geographically disparate technical staff who rotate primary on-call responsibilities. We will thoughtfully engineer systems which permit N+1 failsafes in the unlikely event that multiple technical staff are unreachable”.

Brooke has 6 months experience in setting up and running a node for a different cryptocurrency. Rico and Perly will ensure both Brooke and Dan are trained as fail safe administrators. At this point, the use of external administrators is dependent on how the needs of our business expands and we are confident within the 34.5 yrs of combined experience and capabilities of this core team.
Thx for the clarification. My opinion and experience (but that is my own ;) ). Please refrain from doing that. Monitoring also by less technical inclined? Sure, bonus points.

Having people with the touch of a button taking actions on infrastructure that is fragile currently at best and requires coordination between several parties and immediate action if something goes wrong? Please refrain from doing that the next several years ;) Waiting for a disaster to happen.

Do like that you guys have thought about it though. :)
 
Thx for the clarification. My opinion and experience (but that is my own ;) ). Please refrain from doing that. Monitoring also by less technical inclined? Sure, bonus points.

Having people with the touch of a button taking actions on infrastructure that is fragile currently at best and requires coordination between several parties and immediate action if something goes wrong? Please refrain from doing that the next several years ;) Waiting for a disaster to happen.

Do like that you guys have thought about it though. :)
(y)
 
Thx for the compliment, appreciated.

Well you can take me down the rabbit hole pretty extensively. Have some knowledge about server systems ;)

What type(s) of firewall will you be using? What will you be securing?
Could you list some techniques on the OS level you will be taking to secure the server?

Just some additional information about the above is sufficient.
You will probably get that the Auth nodes and it's security play a vital role in providing the thrust in a decentralized system, that is not fully decentralized
What type(s) of firewall will you be using?

IPtables on linux, Proxmox fw for internal SDN, Cloudflare or Google app armor on the edge to protect from tcp/ip or dns attacks. Iptables on machines to assure they're only speaking to one another with only the guard servers vlan taking app traffic from internet which will in turn pass through software defined network in proxmox to auth. We are looking at utilizing Argo tunnel by cloudflare but pricing and performance have to be tested. Argo is a novel new utility from Cloudflare that would permit our machines to not have any externally listening services whatsoever, and tunnel all traffic through Cloudflare's network via a daemon. This would of course have to be tested but may achieve an unparalleled level of security. Will utilize tools like nmap, thousandeyes, pingdom to do multi-zoned and multi-provider monitoring and alerting [ThousandEyes to keep an eye out for BGP attacks, for instance].

What will you be securing?

non-exhaustively: The host os (core or arch) installation [md5 hash verification] & configuration [hardening init scripts], proxmox software installation & network configuration, guest OS installation & configuration, factomd installation and configuration, key(s) storage and transfer, Hetzner account access, Cloudflare account access, WAF configurations, backups, internal communications, documentation, and processes.

Could you list some techniques on the OS level you will be taking to secure the server?

Hardening by reduction; removing extraneous kernel modules or drivers, enabling SELinux, running only necessary software, disabling password based authentication for SSH [and root login], separate volumes for /tmp, using sudo, using fail2ban
 
One of the challenges we will eventually face be marketing endeavors by different community members/AS teams stepping on each other's toes/not having an aligned message. How would you suggest we handle this? Thanks
One of the core elements of Matter of Fact is collaboration with the Authority Set and the Factom community. To become that integrated community marketing voice.

In terms of teams not stepping on each others toes, the only way I see a balance to this is to by being transparent on what our direction/message is from Matter of Fact and the direction collectively from within the community. MoFs FactomWiki is open sourced, offering a transparent community collection of information.

If elected to the Authority set, we will continue to broadcast this collaboration element of our business to unify an aligned message.
 
Last edited:
Can you prioritize your marketing endeavors for me? Maybe break them out into buckets (short term, medium term, long term) or something like that? Thanks
Hi Matt, Brooke here. Happy to hear you like the mock website and FactomWiki :)

Short term - our focus will be on collating and sharing news from authority nodes, Factom and the community. It's important for the site to be populated with articles before reaching too wide.

Alongside we will be working on strategic SEO articles to gain some organic traffic traction specific to Factom and use cases, etc. This type of traffic takes time and realistically need three months to begin to see some results.

Mid term - once we have some decent traffic and start to be recognised as an authority for Factom news, this is when we can branch out into use case industries to widen exposure and work with an event schedule.

Long term - will be fluid depending on where the Factom protocol is. We will continue items in short and mid-term and have some grandiose ideas, budget allowing. Including applying for grants within and outside the Factom Community to elevate our marketing reach to some of the large crypto, business and tech publications.
 
What type(s) of firewall will you be using?

IPtables on linux, Proxmox fw for internal SDN, Cloudflare or Google app armor on the edge to protect from tcp/ip or dns attacks. Iptables on machines to assure they're only speaking to one another with only the guard servers vlan taking app traffic from internet which will in turn pass through software defined network in proxmox to auth. We are looking at utilizing Argo tunnel by cloudflare but pricing and performance have to be tested. Argo is a novel new utility from Cloudflare that would permit our machines to not have any externally listening services whatsoever, and tunnel all traffic through Cloudflare's network via a daemon. This would of course have to be tested but may achieve an unparalleled level of security. Will utilize tools like nmap, thousandeyes, pingdom to do multi-zoned and multi-provider monitoring and alerting [ThousandEyes to keep an eye out for BGP attacks, for instance].

What will you be securing?

non-exhaustively: The host os (core or arch) installation [md5 hash verification] & configuration [hardening init scripts], proxmox software installation & network configuration, guest OS installation & configuration, factomd installation and configuration, key(s) storage and transfer, Hetzner account access, Cloudflare account access, WAF configurations, backups, internal communications, documentation, and processes.

Could you list some techniques on the OS level you will be taking to secure the server?

Hardening by reduction; removing extraneous kernel modules or drivers, enabling SELinux, running only necessary software, disabling password based authentication for SSH [and root login], separate volumes for /tmp, using sudo, using fail2ban
Thx for the extensive answer :).

Could you elaborate on the WAF? What would you be securing with the WAF?
What would fail2ban monitor?
 
Welcome! :)

Web application firewall would be either Cloudflare or Google Cloud Armor. Would protect management URLs and ingress to the server by wrapping in 2fa and IP whitelisting. Fail2ban would be less required/useful than usual in this instance but rather serve as an aggressive log monitor to ensure no multiple login attempts are being made [i.e. brute force attempts to login on server; such attempts shouldn't be possible with something like Argo so the fail2ban service would be configured to alert/block on a much lower attempt threshold than usual as a last ditch protection scheme].


[I understand that the WAF doesn't eliminate the need for guard servers or protect against application level DoS :) ]
 
Top