Cover photo for Kumar Thangudu
Designing and Launching a Smart Modular Power Distribution Unit for High Performance Compute [Image]Beyond Cost Savings: Ensuring Operational Integrity By implementing smart PDUs, data centers can get closer to the silicon in a sense and ensure operational integrity and prevent similar incidents.  A report by the Ponemon Institute highlighted that The average cost of a severe data center outage exceeds $100,000.per incident [3]. Power-related issues are a major contributor to these costs with primary causes including power failures and inefficient energy management, both of which can be mitigated with smart PDUs [3]. [Image]Conclusion:  High-density compute environments that are grappling with the complexities of GPU servers or crypto mining have a need for a robust smart PDU.  These tools are not just about cost savings; they are about ensuring the reliability and efficiency of data center operations remotely and while ensuring modularity in the deployment.  References:  1. Uptime Institute Annual Outage Analysis 2022 1. Data center outages are decreasing but remain costly, with over 50% of severe outages costing more than $100,000, and 16% exceeding $1 million. 2. Power disruptions are a leading cause of these outages. 3. Uptime Institute Annual Outage Analysis 2022  2. Gartner Report on Data Center Infrastructure 1. The adoption of smart PDUs is expected to grow significantly, driven by the need for better power management and efficiency. 2. Smart PDUs provide remote monitoring, power cycling, and enhanced energy efficiency. 3. 2022 Gartner Report on Data Center Infrastructure 3. Ponemon Institute Cost of Data Center Outages 2021* 1. The average cost of a severe data center outage exceeds $100,000. 2. Power-related issues are a major contributor to these costs. 3. Ponemon Institute Cost of Data Center Outages 2021 4. Data Center Knowledge Article on Power Management** 1. Real-time monitoring systems are crucial for optimizing power distribution and preventing outages. 2.  Inadequate power monitoring can lead to significant operational disruptions and financial losses. 3. Data Center Knowledge Article on Power Management 5. Uptime Institute Global Data Center Survey 2022** 1. The power-per-rack in data centers has increased significantly, with current ranges between 20 to 40 kW per rack. 2. This increase in power density necessitates advanced power management solutions like smart PDUs. 3. Uptime Institute Global Data Center Survey 2022 6. TechTarget Article on Smart PDUs 1. Smart PDUs offer several advantages over traditional PDUs, including remote monitoring, power cycling, and improved energy efficiency. 2. The adoption of smart PDUs is on the rise due to these benefits. 3. TechTarget Article on Smart PDUs 7. Data Center Frontier Report on Power Distribution 1. Data centers face challenges in managing power distribution due to increasing power demands driven by AI and HPC. 2. Smart PDUs help in addressing these challenges by providing better power management and efficiency. 3. Data Center Frontier Report on Power Distribution  8. IEEE Spectrum Article on Data Center Power Management** 1. Real-time monitoring systems help in reducing energy waste and preventing outages by identifying and addressing power anomalies promptly. 2. IEEE Spectrum Article on Data Center Power Management 9. Data Center Dynamics Article on Power Issues** 1. Power-related issues can lead to severe financial implications, with unplanned outages costing data centers millions of dollars. 2. Implementing smart PDUs can mitigate these costs by providing better power management. 3. Data Center Dynamics Article on Power Issues 10. Network World Article on Data Center Efficiency** 1. Smart PDUs contribute to improved energy efficiency and reduced operational costs in data centers. 2. Network World Article on Data Center Efficiency 11. Uptime Institute Report on Data Center Trends** 1. The adoption of smart PDUs is becoming standard in data centers, driven by the need for better power management and efficiency. 2. Uptime Institute Report on Data Center Trends 12. Gartner Forecast on Data Center Power Management** 1. Gartner predicts significant growth in the adoption of smart PDUs, with data centers increasingly recognizing their benefits. 2. Gartner Forecast on Data Center Power Management 13. Ponemon Institute Report on Data Center Costs** 1. Power-related issues are a major contributor to the high costs of data center outages. 2. Implementing smart PDUs can lead to significant cost savings by optimizing power usage and reducing energy waste. 3. Ponemon Institute Report on Data Center Costs 14. Data Center Knowledge Case Study on Smart PDUs** 1. Case studies have shown that smart PDUs can prevent operational issues and reduce costs in data centers. 2. Data Center Knowledge Case Study on Smart PDUs 15. Uptime Institute Insights on Power Management** 1. Industry experts emphasize the importance of advanced power management in data centers. 2. Smart PDUs play a critical role in enhancing data center operations and reducing the risk of power-related issues. 3. Uptime Institute Insights on Power Management 16. Data Center Dynamics Report on Regulatory Compliance** 1. Data centers must comply with various regulatory requirements related to power management, such as Section 889. 2. Compliance ensures the security and reliability of data center operations. 3. Data Center Dynamics Report on Regulatory Compliance 17. TechTarget Article on Offshore PDUs** 1. Using offshore PDUs can pose risks, including potential security vulnerabilities and lower quality standards. 2. It is crucial for data centers to source reliable, high-quality equipment. 3. TechTarget Article on Offshore PDUs 18. IEEE Article on Data Center Power Density** 1. The power-per-rack in data centers is expected to double in the future, necessitating advanced power management solutions like smart PDUs. 2. IEEE Article on Data Center Power Density 19. Data Center Frontier Insights on Power Distribution** 1. Data centers face challenges in coordinating dense electrical equipment and managing power distribution across compact sites. 2. Smart PDUs help in addressing these challenges by providing better power management and efficiency. 3. Data Center Frontier Insights on Power Distribution 20. Network World Report on Smart PDUs** 1. Smart PDUs are becoming increasingly popular in data centers due to their advanced features and benefits. 2. They provide remote monitoring, power cycling, and enhanced energy efficiency. 3. Network World Report on Smart PDUs
Read newest post →

More recent posts

Repairing California Without Fixing the Ballot Process is Impossible

Over the years many of you have seen me write about how onerous pension, infrastructure, and healthcare debt per taxpayer became a wrecking ball to humanity and particularly so for locales with a lot of it. California is a great example. Browse CalMatters and TruthInAccounting for endless data points. There's A TON of people in tech who want to fix California, but they've misframed the problem. I seek to suggest some simple fixes that if well aimed could wreak havoc on the doom loop maiming the quality of life with high costs and weak public services. TLDR: To begin to fix California, you've got to at least do the following as the ante....anything else is a stopgap solution that remedies the symptom but doesn't treat the underlying cause.  • Raise the amount of signatures required for a ballot proposition to 10% or more of a gubernatorial election or 5% or more of a general election. This is an issue.  • Outlaw paid signatures being gathered by special interest groups that can lie to uninformed individuals to get their interests on the ballot. It's damn near impossible to reverse the outcomes of a passed ballot initiative as the requirements are onerous to do so, not easy, and require significant legal meddling. The repeal process is more aggressive and onerous than the process to push a ballot into place. The unit economics of this mean an infinite doom loop for the state of California. 
Read more →

Commercial Office Doom Loop for Cities

I guesstimate that commercial office property taxes make up a 5% to 15% chunk of a locale's tax revenue. It's something worthy of discussion. Here are the descriptions of the fiber optic cable projects with links to more information. I generated it with AI, I wish they'd hyperlink to their sources, oof.  1. PDSCN (Pacific Data Super Cable Network) - The PDSCN is a high-capacity fiber optic cable system that connects the United States to Asia, providing a critical link for international data traffic. The project involves the installation of thousands of kilometers of undersea fiber optic cables, requiring careful planning and execution to ensure reliable and high-speed data transmission. 2. 2Africa- The 2Africa project is a massive undersea fiber optic cable system that will connect Africa, Europe, and Asia, spanning over 37,000 kilometers. This project aims to provide high-speed internet access to millions of people in Africa and beyond, promoting digital inclusion and economic growth. 3. Equiano-The Equiano project is a subsea fiber optic cable system that will connect Europe, Africa, and Asia, with a total length of over 15,000 kilometers. This project is designed to provide high-capacity and low-latency connectivity, supporting the growing demand for internet services in these regions. 4. Gati Shakti - The Gati Shakti project is a national fiber optic cable network in India, aimed at providing high-speed internet access to rural and urban areas. This project involves the installation of thousands of kilometers of fiber optic cables, requiring careful planning and execution to ensure reliable and high-speed data transmission. 5. Coral Bridge - The Coral Bridge project is a fiber optic cable system that connects the Pacific Islands, providing critical telecommunications infrastructure to these remote communities. This project involves the installation of undersea fiber optic cables, requiring specialized equipment and expertise to ensure reliable and high-speed data transmission. 6. KT-Telin - The KT-Telin project is a joint venture between Korean and Indonesian telecommunications companies to build a fiber optic cable system connecting Korea and Indonesia. This project aims to provide high-capacity and low-latency connectivity, supporting the growing demand for internet services in these regions. 7. Te Waipounamu- The Te Waipounamu project is a fiber optic cable system that connects New Zealand's South Island, providing high-speed internet access to rural and urban areas. This project involves the installation of thousands of kilometers of fiber optic cables, requiring careful planning and execution to ensure reliable and high-speed data transmission. 8. Bulikula and Halaihai- The Bulikula and Halaihai projects are fiber optic cable systems that connect the Pacific Islands, providing critical telecommunications infrastructure to these remote communities. These projects involve the installation of undersea fiber optic cables, requiring specialized equipment and expertise to ensure reliable and high-speed data transmission. 9. Digicel- The Digicel project is a fiber optic cable system that connects the Caribbean region, providing high-speed internet access to island nations. This project involves the installation of undersea fiber optic cables, requiring careful planning and execution to ensure reliable and high-speed data transmission. 10. Humboldt- The Humboldt project is a fiber optic cable system that connects the west coast of South America, providing high-capacity and low-latency connectivity. This project aims to support the growing demand for internet services in these regions, promoting digital inclusion and economic growth. 11. KLI (Kochi)- The KLI project is a fiber optic cable system that connects India's Kochi region, providing high-speed internet access to urban and rural areas. This project involves the installation of thousands of kilometers of fiber optic cables, requiring careful planning and execution to ensure reliable and high-speed data transmission. This is a wrecking ball to working in the office. Few truly high agency software developers love showing up to the office. The offices and the landlords have to accommodate for the future.  Unless you've got meals, gyms, trainer, a sauna, and a ton of dogs, I see no reason people who have left the office will rejoin to have a commute when they can literally work from anywhere.  Furthermore, there's the issue of new and onerous EEOC mandates.....
Read more →

American Dynamism is Hype-tastic and We Won't Be Reshoring at Scale Anytime Soon

TLDR: American Dynamism or American reshoring isn’t coming to a city near you at any level of real big scale. Most American degreed STEM grads cannot pass a PE(professional engineer) exam. As a test to future employees in different companies, I’ve frequently given them JEE questions, they can’t pass and this is India’s college entrance exam to get into the top technical institutes. I’ve done this with students who are graduated from MIT, Google, Stanford, Harvard, etc…. Not all STEM programs are of equal caliber. You’re deluded if you believe otherwise. Asymmetric Trade Agreements All or most of our trade agreements allow other countries to dump goods on the USA. KORUS, NAFTA, CAFTA, TPP, TISA are all just basically acronyms for “let other countries achieve trade surpluses with the USA and they end up exporting way more goods to the USA than we export to them.” Closing Thoughts  The only way to address a problem is to frame it well. One of the mantras in engineering is that you’ll spend most of your time framing and measuring a problem before solving it. I think most of the American Dynamism crowd is well intentioned, but horrifically deluded about how the world works if they’re not talking about any of the above items. We have to reframe the conversation with hard STEM talent that has actually completed industrialist workflows in the past leading the charge.  If you're working on hard tech engineering, feel free to reach out to me. 
Read more →

Principles of Paranoia : Engineering Insights

This post was originally written here and got bookmarked over 500 times, way more than I expected.  For the engineers who wanna build physical things at a large scale in the USA (not using pension money or gov't or VC money)/ ie - being resourceful, here's some brouhaha for ya...... For everyone else, this will put you to sleep........ I have had the privilege of working with squadrons of operations research, applied math, and materials engineering PHDs and more. I'm one of 60 engineers on the planet trained in my specialty (operations research + polymer eng.), 1 of 20 in the USA, and 1 of 3 that has designed, scaled, and optimized 100M+ tonne systems multiple times over. My predecessors in some of the companies were ex-CIA and some advised congress on issues related to nat sec. Learned a lot from them. Things to know/learn/be cognizant of: 1) Objective Functions - declare a specific objective(https://byjus.com/question-answer/what-is-the-objective-function-in-linear-programming-problems/#:~:text=The%20real%2Dvalued%20function%20whose,are%20constraints%20and%20are%20variables….) 2) Constraints (https://leanproduction.com/theory-of-constraints/…)  3) How do you measure reality? (read the book "A measure of reality")  4) Local operating environment variables -- you will always unravel the taxonomy and schema of both competitors benchmarking and the environments themselves iteratively. ie leave room to add more columns of data to your modeling and repeat til you have a full picture of the data model and operating environment.  5) Model for weather, breaks in transportation, and feedstock volatility. (Example, I designed a facility that utilized liquid nitrogen to make a byproduct of a polymer --- the O&G prices and futures and quantity I could buy in influenced the viability of the overall polymer's end use case)  6) "Only the paranoid survive." - You need to have a working memory of thousands of real world technologies and a benchmark of their EROI (Energy return on investment)  7) Always, always, talk to the people on the front lines and incentivize their participation in the outcome. Your models will improve.  8) Test your CFO on combinatorics and prob/stat - figure out of they're a glorified bookkeeper or an actual wrangler. (Can they do regressions?) etc.. 9) Deduce the cost of the Bill of Materials and its supply chain not in terms of dollar costs, but in terms of kilowatt hours - this gives you negotiating room with suppliers and service vendors as you can deduce their relative hard costs. Track back to costs. You'll have to put elbow grease into looking up prices on electricity across your forecasted locations in the supply chain. This one thing will give you a basis of measurement that others cannot wrangle.  10) Model feedstock volatility against the Federal Stress Indexes on the dollar.  11) Understanding feedstock volatilities will take you a lifetime, but you need to with relative certainty have low, medium, or high conviction and know where you sit in that chain.  12) Getting verbally violent pseudorandomly with suppliers when all things are calm will prevent headaches in the future. Keep them active, they will get lazy. This is the annoying cost of doing business. Also monitor their churn.  13) If a vendor is telling you that a machine has a max throughput, they're probably lying to you.  14) Throw anyone out of your org if they call themselves a polymath or obsess over the notion of IQ.  15) An engineer who does not break bread with their technicians or doesn't know them well enough is a recipe for danger int he facility, fix immediately.  16) if you want data across the org, assume you'll need a political or violent (not physically) trojan horse to obtain it.  17) if a data request takes longer than 3 days, then the org has to be ripped apart and fixed from the silicon up so to speak.
Read more →

My 2019 Long Levered Natural Gas Bull Thesis 

Kumar’s Natural Gas Sector Bull Thesis Disclosure: My gambles here from least risky to most risky were $FCG, $UAN, and $TELL. I now believe the nat gas bull thesis is strong, but I believe picking the winners is harder now and the thesis is now mainstream. This is not investment advice.  “"In short, China 1000x'd the number of compressed natural gas vehicles in their country in 15 years. They rely heavily on imported natural gas. America is the mecca of natural gas with 65M homes that have natural gas piped in. We often ask 'who killed the electric vehicle' when we should ask 'who killed the compressed natural gas vehicle."   - Me in early 2019” [Image]Not investment advice.  Tldr: My ongoing thesis is that natural gas consumption / kwh will go up for mankind as a %. Feel free to dm me at twitter.com/datarade or tweet to me there.  I began with this thesis in December 2019 and made my bets right around then to March 2020. Published here in mid-2021-ish.  • Nuclear power plant shut downs, delays, commissionings, regulations  • There are ~440 nuclear power plants on the planet.  • Currently it’s difficult to track, but I estimate that there’s ~40 that are being shut down and deactivated. Would love to get more accurate numbers on this.  • IAE.org OR World Nuclear.org  • Every plant shutdown creates an additional 100M-200M cubic feet of daily natural gas demand.  • 3.8% CAGR - https://www.alliedmarketresearch.com/nuclear-power-plant-and-equipment-market#:~:text=The%20global%20nuclear%20power%20plant,3.7%25%20from%202018%20to%202025 • I believe that regulatory power will act against nuclear energy across the world, everywhere except China.  • In the United States, Only 2 nuclear plants were built in the last 25 years. Frequently Asked Questions (FAQs) - U.S. Energy Information Administration (EIA) .  • Even if publicly gaining popularity, nuclear power plants have a strong NIMBY effect. Often localities do not wish to have nuclear power plants.  Air Quality Problems  • breezometer.com • Viable non-US nation states will provide increasing incentives to clean up air quality issues. Natural gas burns clean.  Natural Gas Vehicle and Station Proliferation  • Egypt • Egypt is going to 1k stations by the end of 2021. They currently have 200.  • Egypt is subsidizing the conversion of 400k vehicles, approximately 1 in 10 vehicles in the country.  • https://www.zawya.com/mena/en/business/story/Egypt_plans_to_operate_1000_natural_gas_fuelling_stations__Minister-SNG_218134824/ • All new cars sold in the country are mandated CNG: https://www.bloomberg.com/news/articles/2020-07-12/egypt-to-require-new-vehicles-to-run-on-natural-gas-sisi-says • India • India is going from 2500 stations currently to 10K in < 8 years.  • https://www.financialexpress.com/industry/govt-targets-10000-cng-outlets-in-five-years/2080336/ • India has 2,713 stations as of January 2021 • 60% of the gasoline and diesel prices are driven by tax.  • CNG is effectively 1.5x-2.4x cheaper than diesel and gasoline right now.  • China  • China went from 6000 CNG vehicles to 6M CNG vehicles from 2000 to 2017.  • 1000x multiple in 17  years. I don’t know the numbers right now on CNG vehicles and stations in China.  • By the end of 2018, ownership of motor vehicles in China reached 327 million units including 240 million cars, and accordingly there were up to 110,000 filling stations and at least 9,000 natural gas stations across the country, according to a new reports by Research and Markets. • https://www.petrolplaza.com/news/21890 • The South China Sea has 180T cubic feet of natural gas.  • China consumes 10T cubic feet of natural gas annually.  Natural Gas Peaking Plant expansion  • CAGR of ~5% • https://www.grandviewresearch.com/industry-analysis/natural-gas-fired-electricity-generation-industry Other • Fertilizer 2.5% CAGR for the next 5 years.  • Renewables Proliferation = more natural gas consumption/kwh • 36 states have substantial renewable goals due by 2025. State Renewable Portfolio Standards and Goals (ncsl.org) • Many states have renewable mandates due in 2025 and 2030. • For the grid to function in a way American’s expect, many states can not truly hit their Renewable mandates without relying on stable sources of energy (such as Nuclear or fossil fuel).  • Out of the big 3 fossil fuel, natural gas emits the least amount of carbon. • To fill in the gap between mandate and reality, states/regulatories may start redefining natural gas as green. • Natural gas could be a major player in the “renewable” space ie how the US’s oldest source of energy “Wood” has been redefined as a renewable source “Biomass”. Biomass explained - U.S. Energy Information Administration (EIA) • Because natural gas is simply hydrocarbon, there are already countless “green” ways to produce such as from food waste, anaerobic digestion etc even without a re-definition.  • Tax Policy - Governments provide discounts to Natural Gas adopters vs. Gasoline + Diesel adopters • Natural Gas Commercial vehicles have lower maintenance costs.  • Natural Gas Pipeline expansion • 3.4% CAGR • https://www.grandviewresearch.com/industry-analysis/gas-pipeline-infrastructure-market#:~:text=Report%20Overview,3.4%25%20from%202020%20to%202027 • The reason the USA has not adopted CNG vehicles at scale is completely politically driven and has little to do with economics or technicals of natural gas. https://news.ycombinator.com/item?id=1045211 • The median/general break even price for natural gas producers is $3.00 per MMBtu Are Natural Gas Prices Below $3 Sustainable? - Enerdynamics Added by people other than Kumar:  • For nearly 2 entire  years (2019 and 2020), natural gas was below the general break even price. And much so in certain months.  • Haynesville break-even is as low as $2.05 with median near $2.25  • Marcellus break- even is as low as $1.55 with median near $1.85 • For the past 6 years, natural gas prices have been near or the break even point. https://www.eia.gov/dnav/ng/hist/rngwhhdm.htm • (opinion) To recoup past losses, suppliers will naturally increase price (or shut down) irrespective of many demand trends for a sustained period of time.  • Natural Gas supply (U.S.) • ~135 Bcf of supply from US (from Rystad Energy).  • Permian operators will be able to ramp gas production (+25 to 35% in next 3 years) with new pipelines going to Mexico, Corpus Christie (LNG export), and local markets • Haynesville operators have increased rig activity and location advantage to Gulf Coast LNG export • Marcellus and Utica (North East) will meet US demand but actively seeking ways to get to Gulf Coast via new gas pipelines • EQT largest independent natgas producer; using electric & dual-fuel fleets to complete wells and others watching to see how the company reduces diesel costs and increase use of “field” natgas • Global: 10 billion boe (barrels of oil equivalent) discovered in 2020  • Risk:  • FERC implementing new policies to restrict pipeline development;  • bitcoin and other crypto miners burning gas (first “stranded” or “to reduce flaring” in permian, perhaps ultimately gas that’d otherwise be economical to take away)  • US Commercial Vehicles, Demand • Logistics continue to grow to increased imports, ecommerce, and short-haul demand • LNG / CNG has better energy density and fits heavy-duty long distance requirements • Dual-fuel engines from CAT are aggressively being adopted by oilfield companies. CAT is deploying across Mining and other industries • Fedex contracted CNG truck fleet for long-haul and electric fleets for short-haul • Top 3 commercial vehicle fleet have ~300,000 trucks/tractor-trailers • LNG Supply / Demand • LNG production is expected to reach 672 Mt in 2040 with US, Middle East, and Mozambique. Good graphic on gas markets supply/demand  • China adding 2-3 LNG terminals and currently has 22 LNG plants able to receive 81 Mt/y • CNPC expected China's pipeline gas imports to reach about 100 Bcm by 2025, nearly double from the 2019 level • Risk: US policy, if Biden limits export capability • Idiosyncratic market • Important to understand that natural gas/LNG market is not analogous to oil market • Difficult to transport and store; high fixed costs of midstream, floating and fixed liquefaction, regas, etc. infrastructure • Mostly producers and end-users drive the market; unlikely there will be a “global benchmark price” or hyper liquid spot/derivatives market that resembles Brent or WTI • Risk: US policy, if Biden limits export capability • Global Natgas Demand • S&P estimates ~5% increase in 2021 • Shell demand estimates • 1% global CAGR in • Asia  • 3% CAGR gas demand • 4% CAGR lng import (600 BCM by 2040; currently ~ 300 BCM) • BP  • statistical review of world energy  • https://www.bp.com/content/dam/bp/business-sites/en/global/corporate/pdfs/energy-economics/statistical-review/bp-stats-review-2021-full-report.pdf • Renewable Natural Gas • Not sure if you include this but methane from landfills have lower carbon intensity • RNG primer - https://www.mjbradley.com/sites/default/files/MJB%26A_RNG_Final.pdf
Read more →