Tech Updates for Busy People, September 2023

Table of Contents

Hello World!

Here are some of the updates that I found interesting for this month, and if you find them useful as well, feel free to share this blog with your family and friends.

The Worst Programmer I Know

Now that McKinsey is claiming they fund the magic formula to measure Developer Productivity, I found this very interesting article from Dan North: The Worst Programmer I Ever Worked With.

And in case you want to go deeper, here is the very interesting response that Gergely Orosz and Kent Beck put together: Measuring developer productivity? A response to McKinsey. I strongly suggest you subscribe to their newsletter.

Kafka is dead, long live to Kafka

Far from being a Kafka expert myself, I found this very interesting project called WarpStream.

OpenTF is now OpenTofu

OpenTF has been renamed OpenTofu and is now part of the Linux Foundation. I guess is safe to say that we have another solid alternative to Terraform. Feel free to check the OpenTofu Github Repository.

Another Microsoft Breach

38TB of data accidentally exposed by Microsoft AI researchers38TB of data accidentally exposed by Microsoft AI researchers. Wiz Research found a data exposure incident on Microsoft’s AI GitHub repository, including over 30,000 internal Microsoft Teams messages – all caused by one misconfigured SAS token.

AWS

AWS Dedicated Local Zones

AWS Dedicated Local Zones are a type of AWS infrastructure that is fully managed by AWS, built for exclusive use by you or your community, and placed in a location or data center specified by you to help comply with regulatory requirements.

Amazon Aurora: Global Database Failover

Amazon Aurora now supports Global Database Failover, a fully managed experience for performing a cross-Region database failover to respond to unplanned events. With Global Database Failover, you can convert a secondary region into the new primary region in typically a minute and also maintain the multi-region Global Database configuration.

Amazon RDS: Performance Insights supports SQL-level metrics for Amazon RDS for SQL Server

Amazon RDS Performance Insights supports SQL-level metrics for Amazon RDS for SQL Server so that you can identify high-frequency, long-running, and stuck SQL queries in seconds. SQL level statistics are already available for all other Amazon RDS engines.

Amazon EMR on EKS now supports managed Apache Flink - Public Preview

Amazon EMR on EKS now supports managed Apache Flink, available in public preview. With this launch, customers who already use EMR can run their Apache Flink application along with other types of applications on the same Amazon EKS cluster, helping improve resource utilization and simplify infrastructure management.

You can now subscribe Amazon Simple Queue Service (SQS) Standard queues to Amazon Simple Notification Service (SNS) First-In-First-Out (FIFO) topics. Thus, from a single SNS FIFO topic, you can now deliver messages to SQS Standard queues, which offer best-effort ordering and at-least-once delivery, as well as to SQS FIFO queues, which support strict ordering and exactly-once delivery. This new capability further decouples message publishers from subscribers, as the SNS topic type no longer dictates the SQS queue type that subscribers ought to use.

AWS IAM provides action last accessed information for more services

AWS Identity and Access Management (IAM) now provides action last accessed information for more than 140 services to help you refine the permissions of your IAM roles. You can review action last accessed information, identify unused permissions, and refine to scope down the access of your IAM roles to only the actions that they use for services such as Amazon CloudWatch, AWS Key Management Service (AWS KMS), and Elastic Load Balancing (ELB).

Amazon GuardDuty introduces cluster configurability in EKS Runtime Monitoring

This new capability allows you to selectively configure which Amazon Elastic Kubernetes Service (Amazon EKS) clusters are to be monitored for threat detection. Previously, configurability was at the account level only. With this added cluster-level configurability, customers can now selectively monitor EKS clusters for threat detection or continue to use account level configurability to monitor all EKS clusters in a given account and region.

AWS Elastic Disaster Recovery: post-launch actions framework

AWS Elastic Disaster Recovery (AWS DRS) now allows you to define actions that run automatically after launching recovery instances, empowering you to automate any action you need to run after an instance has successfully launched.

Cost Anomaly Detection increases custom anomaly monitor limit

Cost Anomaly Detection uses machine learning to continuously monitor, detect, and alert customers of anomalous spent pattern. Starting today, Cost Anomaly Detection users with a management account will be able to create up to 500 custom anomaly monitors to track spend in their account(s). A custom anomaly monitor allows a user to track AWS spend across either linked accounts, cost allocation tags, or cost categories.

Amazon SageMaker now provides a new quick Studio setup experience

The new quick Studio setup experience for Amazon SageMaker provides a new onboarding and administration experience that makes it easy for individual users to setup and manage SageMaker Studio. Today, individual users, who are new and/or just looking for a basic managed notebook experience on the cloud, need to understand and configure concepts like IAM, domains, VPC first to create their Studio environment.

Amazon RDS Custom for SQL Server lets you stop and start your database instance on demand

Amazon RDS Custom for SQL Server makes it simple to stop and start your database instances. This lets you save costs by stopping a database instance when it does not need to be running, such as when it is used for development or test purposes.

Azure

Preview: OpenAI Whisper model in Azure OpenAI service and Azure AI Speech

Azure OpenAI Service and Azure AI Speech now offer the OpenAI Whisper model in preview. The OpenAI Whisper model has multi-lingual capabilities that offer precise and efficient transcription of human speech in 57 languages, and translation into English. It also creates transcripts with enhanced readability.

Authenticate Azure Monitor logs connector in Logic App with managed identity

Managed identities in Azure lets you authenticate to any resource that supports Azure AD authentication, including your own applications. When you enable managed identity authentication in Logic App and grant it permissions in Log Analytics workspace or Application Insights component, you can query data without needing to provide credentials, secrets, or Azure AD tokens, for Azure Monitor Logs connector authentication.

Preview: Alerts timeline view

Azure Monitor alerts is previewing a new timeline view that simplifies the consumption experience of fired alerts. The new view has the following advantages:

Shows fired alerts on a timeline
Helps identify co-occurrence of alerts
Displays alerts in the context of the resources they fired on
Focuses on showing counts of alerts to better understand impact
Supports viewing alerts by severity
Provides a more intuitive discovery and investigation path

Give the new experience a try by selecting “View as timeline (preview)” from the Alerts command bar.

Malware Scanning in Defender for Storage is now GA

(Malware Scanning in Defender for Storage is Generally Available)[https://techcommunity.microsoft.com/t5/microsoft-defender-for-cloud/malware-scanning-for-cloud-storage-ga-announcement-prevent/ba-p/3884470]. Malware Scanning helps overcome traditional challenges in non-compute malware protection, providing a scalable solution tailored for high-compliance industries. Malware Scanning is available as an add-on to Defender for Storage and is a significant enhancement to Microsoft Defender for Cloud’s security offerings for Azure Blob Storage.

Azure Machine Learning - Public Preview for September

Three new features now available in Public Preview enable you to utilize vision-based models in the model catalog, enjoy seamless integration with LLMOps through the Prompt Flow SDK/CLI and VS Code extensions, and monitor LLM completions for safety and quality metrics.

Vision models for model catalog: You can now utilize new task types, namely image classification, image segmentation, and object detection, to fine-tune, evaluate, and deploy base models using your own training data.
Enjoy the Prompt Flow code-first functionality: You can now fully leverage the power of LLMs, start quickly with pre-built workflows and evaluation metrics, and enjoy high-efficiency iterations with LLMOps integration into Prompt Flow.
Model Monitoring for LLM Applications: You can now monitor LLM applications in production for safety and quality on a user-defined cadence to ensure they deliver maximum business impact.

Preview: Save Azure Backup Recovery Services Agent (MARS) passphrase to Azure Key Vault

Now, you can save your Azure Recovery Services Agent encryption passphrase in Azure Key Vault directly from the console, making the Recovery Services Agent installation seamless and secure.

Higher default value for Azure Functions Event Hubs max batch size

Starting with version 6.0.0 of the Azure Functions Event Hubs extension, a higher default value of 100 will be used for the maxEventBatchSize configuration setting. This setting represents the maximum number of events from Event Hubs that the function can receive when it’s invoked. This matches the defaults of the Azure SDK for Event Hubs and will apply starting with version 6.0.0 of the Event Hubs extension.

You can now use Azure Key Vault to securely store and retrieve Azure Storage account access key when mounting it as a local share in App Service. Azure Key Vault has the benefits of storing application secrets centrally and securely with the ability to monitor, administer, and integrate with other Azure services like Azure App Service.

This feature is available for App Service Linux code and container, Windows containers and Windows code.

Preview: Configure customer-managed keys on existing Cosmos DB accounts

You can now enable Customer Managed Keys (CMK) on existing Azure Cosmos DB accounts. This eliminates the need to migrate data to a new account to enable CMK.

This release follows the existing ability to enable a second layer of encryption for data at rest using CMK while creating a new Azure Cosmos DB account.

Azure SQL updates for mid-September 2023

In mid-September 2023, the following updates and enhancements were made to Azure SQL:

Verify the automated backups taken in Azure SQL Database using sys.dm_database_backups. Learn more.
Keep your Azure SQL Database Hyperscale backups for up to 35 days of point-in-time retention. Learn more.
Keep a copy of your Azure SQL Database Hyperscale backups in long-term retention (LTR) for up to 10 years. Learn more.

Latest generation burstable VMs - Bsv2, Basv2, and Bpsv2 now Generally Available

The Bsv2, Basv2, and Bpsv2 series virtual machines are the latest generation of Azure burstable general purpose VMs, providing a baseline level of CPU utilization and capable of expanding to higher CPU utilization as workload volume increases. This is ideal for many applications such as development and test servers, low traffic web servers, small databases, micro services, servers for proof-of-concepts, build servers, and code repositories.

Export Cost Management data to secure storage accounts with firewall are now Generally Available

You can export Cost Management data to Azure storage accounts with firewall. Users can utilize either the Exports API or the Azure portal to create recurring tasks for the automatic export of cost data in CSV format. This can be scheduled on a daily, weekly, or monthly basis, and the exported data can be used for the creation of dashboards or integration with financial systems.

View trends of client-side metrics across test runs in Azure Load Testing

Using Azure Load Testing, you can now track trends in client-side metrics from your last ten test runs. This helps you see if your performance is getting better or worse.

You can also pick one test run as a baseline and compare the recent metrics to it. This way, you can tell if your performance meets your expectations. More info here.

And additionally, you can now Configure load testing in your CI/CD pipeline from Azure portal.

Azure Front Door Standard and Premium support bring your own certificated based domain validation

Azure Front Door Standard and Premium support Bring Your Own Certificates (BYOC) based domain ownership validation. Azure Front Door automatically approves the domain ownership if the Certificate Name (CN) or Subject Alternative Name (SAN) of the provided certificate matches the custom domain and the certificate is valid. This feature reduces the steps and efforts required to prove the domain ownership and makes the Dev-Ops experience more streamlined.

Preview: Azure Container Storage Updates: Expanded regions and performance options

With Azure Container Storage in preview, you can easily use block storage volumes for production-scale stateful container applications on Azure Kubernetes Service (AKS). Azure Container Storage provides rapid scale out of volumes, reduced pod failover time, reduced total cost of ownership, and consistent access to local & remote storage. More information here.

Azure Monitor VM Insights using Azure Monitor Agent is now Generally Available

Azure Monitor VM Insights is now generally available using the Azure Monitor Agent. VM Insights provides a quick and easy method to monitor the client workloads on your Azure virtual machines and virtual machine scale sets as well as Azure Arc enabled servers running in an on-premises/multi-cloud environment.

Azure Firewall: Explicit Proxy is now in public preview

Explicit Proxy (public preview): Azure Firewall now supports Explicit proxy mode on the outbound path. With this mode enabled, you have the option to configure a proxy setting directly on the sending application, such as a web browser, with Azure Firewall acting as the designated proxy. This configuration allows traffic from the sending application to be directed to the private IP address of the firewall, facilitating direct egress from the firewall without the need for a UDR.

Retirement: Kubernetes 1.24 support in AKS

Kubernetes version 1.24 support in AKS is now deprecated. With this deprecation, AKS will no longer support 1.24 and the version is moving to platform support. If you are currently using version 1.24, you should migrate to Kubernetes version 1.25 or higher.

For additional information, visit: Supported Kubernetes versions in Azure Kubernetes Service (AKS)

Preview: Rate-limit rules for Application Gateway Web Application Firewall

Azure’s regional Web Application Firewall (WAF) running on Application Gateway now supports rate-limit custom rules. Rate-limiting enables you to detect and block abnormally high levels of traffic destined for your application. By using rate limiting, you can mitigate many types of denial-of-service attacks, protect against clients that have accidentally been misconfigured to send large volumes of requests in a short time period, or control traffic rates to your site from specific geographies.

Azure Container Apps Enhancements

Supports for Environment level mTLS encryption:

Azure Container Apps now supports environment level network encryption using mutual transport layer security (mTLS). This feature is in preview.

When end-to-end encryption is required, mTLS will encrypt data transmitted between applications within an environment.

Azure Container Apps Jobs are now Generally Available

Jobs are now generally available.

In addition to continuously running services that can scale to zero, Azure Container Apps now supports jobs. Jobs enable you to run serverless containers that perform tasks that run to completion.

Azure Container Apps jobs support three trigger types: manual (on-demand), scheduled, and event-driven. Manual jobs are triggered by a user or an external system, such as another container app. Scheduled jobs are triggered at a specified time or interval. Event-driven jobs are triggered by scaling rules.

Azure Container Apps supports additional TCP ports

Azure Container Apps now support additional TCP ports, enabling applications to accept TCP connections on multiple ports. This feature is in preview.

Azure Container Apps support for UDR, NAT Gateway, and smaller subnets is now Generally Available

User defined routes (UDR), NAT Gateway, and smaller required subnet sizes are now generally available for Azure Container Apps on the new (workload profiles)[https://learn.microsoft.com/en-us/azure/container-apps/networking?tabs=azure-cli#environment-selection] environment type. Workload profiles environments support both the consumption and dedicated plans.

Generally available: Azure Container Apps dedicated plan

Azure Container Apps dedicated plan is now generally available in the new workload profiles environment type. A workload profile determines the amount of compute and memory resources available to container apps deployed in an environment.

Compute options are represented as workload profiles defined at the Azure Container Apps environment scope. We currently support general purpose and memory optimized workload profiles with up to 32 vCPU’s and 256 GiB’s of memory.

Google Cloud Platform

Cloud Database Migration Service

Database Migration Service now supports customer-managed encryption keys (CMEK) that are externally managed with Cloud External Key Manager. For more details on CMEK support for each migration scenario.

Cloud Logging

You can now save charts generated from a Log Analytics SQL query to a custom dashboard. For more information.

Compute Engine

Snapshot settings are centralized configuration parameters for all snapshots in a project. You can use snapshot settings to customize the default storage location for all future snapshots in your project. By enabling you to do this, snapshot settings remove the need for you to manually specify a storage location during each individual snapshot creation.

Google Kubernetes Engine

GKE clusters running version 1.28 or later block new bindings of ClusterRole cluster-admin to User system:anonymous, Group system:authenticated, or Group system:unauthenticated due to the security risks of these bindings. GKE does not block existing bindings.

Compute Engine persistent disk CSI Drivers deployed on clusters running version 1.26 and later now support filesystem size expansion during restoration from a snapshot or a clone when the PVC data source is larger than the original volume.

AlloyDB for PostgreSQL

Maintenance operations on highly available primary instances now occur with less than one second of downtime for most workloads.

You can now configure SSL enforcement mode to ensure that all database connections to an instance use SSL encryption.

When creating an AlloyDB cluster, you can now specify an IP range for private services access. This is optional; if you do not specify an IP range, then AlloyDB selects one for you.

Virtual Private Cloud

Policy-based routing is available in General Availability. You can select a next hop based on more than a packet’s destination IP address. You can match traffic by protocol and source IP address as well.

Capacity Planner

You can view the following when using Capacity Planner:

The 50th and 75th percentile usage and forecast of your VMs.
The historical usage of your VMs up to 2 years in the past.
The usage and forecast of all machine families in a project.