Tech Updates for Busy People, October 2023

Table of Contents

Hello World!

Quite a loaded entry today!
Here are some of the updates that I found interesting for this month, and if you find them useful as well, feel free to share this blog with your family, friends and pets.

Private Estimates, Public Progress

Great entry from Kent Beck about planning, deadlines, and team workflow Private Estimates, Public Progress.

Checklist: Is your application ready for a container cluster?

An old post that alway comes handy when you are planning to deploy a new application, or modernizing existing ones Checklist: Is your application ready for a container cluster?.

AWS Cost Optimization 101

Another good and old but still relevant post about AWS cost optimization AWS Cost Optimization 101

Lowering our AWS RDS spend using Aurora I/O-Optimized

An excellent entry on rewind’s engineering blog about how reading the fine print and correctly understanding how your workload behaves can save you hundreds of dollars Lowering our AWS RDS spend using Aurora I/O-Optimized.

AWS

Application Load Balancer and Network Load Balancer now support registering instances addressed by IPv6 as targets

IPv6 instance type targets support on ALB and NLB is available in all commercial AWS Regions, the AWS GovCloud (US) Regions and Outposts. To learn more, please refer to the ALB documentation and NLB documentation.

Amazon EC2 Hibernate now supports more operating systems

Amazon Elastic Compute Cloud instances (Amazon EC2 instances) now support Hibernation for the Microsoft Windows Server 2022, Red Hat Enterprise Linux 9, and Amazon Linux 2023 operating systems. Hibernation is an Amazon EC2 feature that helps lower costs and achieve faster startup times by enabling customers to pause and resume their running instances at scale. Customers are not billed for compute time while instances are suspended, and your applications will resume from right where they left off. More information about Hibernation can be found on the Amazon EC2 User Guide.

Lambda test events are now available in AWS SAM CLI

This capability makes it easier for developers using SAM CLI to collaborate and streamline testing workflows. It also allows developers to use a consistent set of test events across their entire team. Developers can make test events available to other team members in their AWS account using granular IAM permissions. More information here

New updates to the AWS Well-Architected Framework

Not a lot more to say, go read the updates here.

Amazon EventBridge announces support for wildcard filters in rules

Amazon EventBridge rules now support wildcard filters, which enable you to match any character or sequence of characters within a string in your event payload. For example, you can use wildcards to match against values that end in a specific file type in a directory, such as “dir/*.png”, or contain a specific word, such as “AcmeCorp”. Learn more about this new feature here.

Amazon EKS Extended support for Kubernetes Versions now available in preview

Amazon EKS follows the Kubernetes project release cycle. For the first 14 months after a minor version is released as generally available, it is covered under standard support. Now, Amazon EKS Extended Support gives 12 additional months of support for each Kubernetes minor version, for a total of 26 months of support.

Amazon SageMaker Feature Store in-memory online store for low latency feature retrieval

Amazon SageMaker Feature Store now supports a fully managed, in-memory online store, which enables you to retrieve features for model serving in real time for high throughput ML applications. The new online store is powered by ElastiCache for Redis, which is a blazing-fast in-memory data store built on open-source Redis.

Amazon DataZone is now generally available

This data management service is designed to catalog, discover, analyze, share, and govern data at scale across organizational boundaries with governance and access controls. It provides data visibility and helps data producers and consumers across business units more securely share data. To learn more, see the Amazon DataZone webpage.

Amazon SageMaker Model Registry announces support for private model repositories

Amazon SageMaker Model Registry now allows you to register machine learning (ML) models that are stored in private Docker repositories. This capability enables you to track all your ML models across multiple private AWS and non-AWS model repositories in one central service to simplify ML operations (MLOps) and ML governance at scale. For more details on how to deploy such models from private repositories, refer to the developer guide.

AWS Firewall Manager supports referencing of Security Groups

AWS Firewall Manager now supports referencing of security groups as part of its security group common policies. With this feature, customers can update the inbound or outbound rules for the Firewall Manager primary security groups to reference security groups in the peered VPC. This allows traffic to flow to and from instances that are associated with the referenced security group in the peered VPC. For more information, visit the AWS Firewall Manager webpage.

AWS IoT Device Management announces general availability of Software Package Catalog

With this new feature, you can now track and monitor software package versions across fleet, gain valuable insights from a central dashboard, and perform targeted updates on devices running on a specific software version. More information here.

AWS Control Tower releases 22 proactive controls and 12 AWS Security Hub Detective controls

This release increases the range of controls in AWS Control Tower controls library with the addition of controls for services such as Amazon Athena, Amazon EMR, and Amazon Neptune. Check Control Tower’s documentation for more information.

New AWS Network Load Balancer (NLB) availability and performance capabilities

AWS Network Load Balancer (NLB) now supports Availability Zone DNS affinity, disables connection termination for unhealthy targets, and UDP connection termination by default. More info here, here, and here.

CloudWatch launches out-of-the-box alarm recommendations for AWS services

Amazon CloudWatch announces out-of-the box, best practice alarm recommendations for AWS service-vended metrics. It provides alarm recommendations and alarm configurations for key vended metrics, along with the ability to download pre-filled infrastructure-as-code templates for these alarms. Furthermore, you can now see in-line descriptions for AWS service metrics across the AWS console, which enables you to easily see metric details to help you troubleshoot or assess system health. More info here.

Amazon Relational Database Service announces Dedicated Log Volume

Amazon Relational Database Service (Amazon RDS) now supports a Dedicated Log Volume for PostgreSQL, MySQL, and MariaDB databases. An Amazon RDS Dedicated Log Volume allows customers to select a configuration where the most latency-sensitive components of their database, the transaction logs, are stored in a separate, dedicated volume. Dedicated Log Volumes work with Provisioned IOPS storage and are recommended for databases with 5,000 GiB or more of allocated storage.

Enable Amazon RDS Optimized Writes using RDS Blue/Green Deployments

Amazon Relational Database Service (Amazon RDS) Optimized Writes can now be enabled for your current RDS for MySQL and RDS for MariaDB database instances and promoted to production in as fast as a minute using RDS Blue/Green deployments. RDS Optimized Writes can improve up to 2x write throughout for your databases. Learn more about Blue/Green Deployments on the Amazon RDS features page.

AWS Service Catalog announces support for additional Infrastructure as Code (IaC) provisioning tools

AWS Service Catalog customers can now create, distribute, and launch AWS resources that are configured using third-party Infrastructure as Code (IaC) tools such as Ansible, Chef, Pulumi, Puppet, and more. Within AWS Service Catalog, customers can use these IaC tools in addition to previously supported AWS CloudFormation and HashiCorp Terraform Cloud configurations. To learn more about using this feature, go here.

Amazon EKS now allows modification of cluster subnets and security groups

AWS Customers can now update the subnets and security groups associated with their existing Amazon Elastic Kubernetes Service (EKS) clusters. This additional cluster management flexibility makes it simpler for cluster administrators to stay in sync with changes made to Amazon Virtual Private Cloud (VPC) resources.

Amazon SNS message data protection now supports custom data identifiers

Amazon SNS message data protection is a set of capabilities that leverage pattern matching, machine learning models, and content policies to help security and engineering teams facilitate real-time data protection in their applications that use Amazon SNS to exchange high volumes of data. Now, you can use custom data identifiers to detect the protect domain-specific sensitive data, such as your company’s employee IDs. Previously, you could only use managed data identifiers to detect and protect common sensitive data, such as names, addresses, and credit card numbers.

AWS Trusted Advisor adds 64 new checks powered by AWS Config

AWS Trusted Advisor adds a new operational excellence check category and integrates with AWS Config to deliver 64 new best practice checks across all categories. Trusted Advisor continuously evaluates your AWS environment using best practice checks in the categories of cost optimization, performance, resilience, security, operational excellence, and service limits, and recommends actions to remediate any deviations from best practices. The new best practice checks are powered by AWS Config managed rules which are predefined, customizable rules used to evaluate the compliance state of your AWS resources.

Azure

Default Rule Set 2.1 for regional WAF with Application Gateway

Default Rule Set (DRS) 2.1 for the regional Web Application Firewall (WAF) on Azure Application Gateway is now generally available on the Azure Application Gateway WAF V2 SKU.

DRS 2.1 is baselined off the Open Web Application Security Project (OWASP) Core Rule Set (CRS) 3.3.2 and extended to include additional proprietary protection rules developed by Microsoft Threat Intelligence team. The Microsoft Threat Intel team analyzes Common Vulnerabilities and Exposures (CVEs) and further adapts the CRS ruleset to address CVEs and reduce false positives. More information here

Public preview: Announcing the new Azure Bastion Developer SKU

The Bastion Developer SKU represents a novel, cost-effective, and streamlined version of the Bastion service. Bastion Developer allows users to establish secure connections to a single VM at a time without the necessity of additional network configurations or exposing public IPs on VMs.

Azure Blob Storage Cold Tier support on Blob Batch operations is now GA

With Blob Batch operations, you can change tier of massive blobs to or from a blob access tier with optimal performance. It now supports the tiering operations for cold tier. Refer to the documentation for additional details.

Public preview: Traffic Splitting for Azure Static Web Apps

With traffic splitting, you can split the traffic of your Azure Static Web Apps resource across multiple environments and specify the amount of traffic each environment should receive. This enables you to gradually roll out new changes to your site which can mitigate risks of new changes. Traffic splitting can also be used to implement A/B testing which enables data-driven decision making and facilitates optimizations.

AKS

General availability: Storage auto-grow & online disk scaling for Azure Database for PostgreSQL - Flexible Server

With auto-grow enabled, Azure Database for PostgreSQL - Flexible Server will automatically increase the size of the provisioned storage of your database servers. This flexibility removes concerns about rightsizing when beginning to use these services and running out of storage.

Public preview: Flush data operation for Azure Cache for Redis

The flush data operation for Basic, Standard and Premium caches allows you to delete or flush all data in your cache. This flush operation can be used before scaling operations to potentially reduce the time required to complete the scaling operation on your cache. You can also configure the flush operation to run periodically on your dev/test caches to keep memory usage in check.

Now you can enable Azure Synapse Link on your existing MongoDB collections stored in Azure Cosmos DB for Mongo DB. Unblock near real-time analytics at no RU/s cost.

General availability: PostgreSQL 16 production support in Azure Cosmos DB for PostgreSQL

You can now select the latest major PostgreSQL version, PostgreSQL 16, for Azure Cosmos DB for PostgreSQL clusters. Existing clusters can be upgraded to PostgreSQL 16 using the in-place upgrade feature. With PostgreSQL 16 available on Azure Cosmos DB for PostgreSQL clusters, you can benefit from: query performance improvements with more parallelism; developer experience enhancements; monitoring of I/O stats using pg_stat_io view; and enhanced security features, among other improvements.

The availability of Azure compute reservations exchanges extended until at least July 1st, 2024

Through a grace period, you will have the ability to exchange Azure compute reservations (Azure Reserved Virtual Machine Instances, Azure Dedicated Host reservations, and Azure App Services reservations) until at least July 1, 2024.

Generally available: Zone Redundant Storage for Azure Disks is now available in more regions

Zone Redundant Storage (ZRS) for Azure Disk Storage is now generally available on Azure Premium SSDs and Standard SSDs in Norway East and UAE North regions.

Default outbound access for VMs in Azure will be retired

Azure is moving towards a secure-by-default model. This means default outbound access to the internet will be turned off. After 30 September 2025, Azure will no longer assign a default implicit IP for VMs to communicate to the internet. Existing VMs will not be impacted by this retirement. For new workloads, you need to transition to an explicit outbound method.

General Availability: Azure AI Content Safety

Azure AI Content Safety, now generally available, is a service that enables developers to build safer online environments by detecting and assigning severity scores to unsafe images and text across content categories and languages.

Public Preview: VMSS Automatic Instance Repairs - Reimage, Restart Repair Actions

Automatic instance repairs help Virtual Machine Scale Set customers achieve high application availability by automatically detecting and recovering unhealthy VM instances at runtime.

Azure Machine Learning

Three features now available in GA:

Four features now available in Public Preview:

  • Environment Support on Compute Instance: You can now use the same image when running a full job on a cluster or an experiment on a compute instance.
  • Model Packaging (v2): You can now build model packages to deploy to Online Endpoints through the Azure Machine Learning inference server or a custom inference server of your choice.
  • Label pixels in images through Semantic Segmentation: You can now add tags or labels to individual pixels within images, leverage the vendor workforce in labeling the images, and label categories through hierarchical labeling.
  • New base inference models with fine-tuning capabilities: You can now utilize two new base inference models (Babbage-002 and Davinci-002) and fine-tuning capabilities for three models (Babbage-002, Davinci-002, and GPT-3.5-Turbo). We are adding fine-tuning capabilities to these three models and making them accessible through the Azure Machine Learning model catalog.

Azure Dedicated Host - Resize

With Azure Dedicated Host’s new ‘resize’ feature, you can easily move your existing dedicated host to a new Azure Dedicated Host SKU (e.g., from Dsv3-Type1 to Dsv3-Type4). This new ‘resize’ feature minimizes the impact and effort involved in configuring VMs when you want to upgrade your underlying dedicated host system.

ExpressRoute Traffic Collector is now generally available

ExpressRoute Traffic Collector enables you to capture information about IP flows sent over ExpressRoute direct circuits. You can enable flow logs capture for both Private and Microsoft peering with ExpressRoute Traffic Collector. Captured flow logs data get sent to a Log Analytics workspace where you can create your own log queries for further analysis.

General Availability: Upgrade your existing Azure Gen2 VMs to Trusted Launch

Support for upgrade of existing Azure Generation 2 VMs to Trusted Launch is now generally available. You can seamlessly enable Trusted Launch and improve the security of your existing Azure Generation 2 VMs. Trusted Launch VMs provide foundational compute security to Azure Generation 2 VMs by enabling Secure Boot and vTPM capabilities. Trusted Launch capabilities protects OS against rootkits, boot kits and enables attestation by measuring the boot chain of VM.

Azure Container Apps is now eligible for Azure savings plan for compute

Azure Container Apps is now eligible for Azure savings plan for compute! All Azure Container Apps regions and plans are eligible to receive 15% savings (1 year) and 17% savings (3 years) compared to pay-as-you-go when you commit to an Azure savings plan.

Google Cloud Platform

GCP’s published Storage decision tree helps you research and select the storage services in Google Cloud that best match your specific workload needs and the accompanying blog provides an overview of the services offered for block storage, object storage, NFS and Multi-Writer file storage, SMB storage, and storage for data lakes and data warehouses.

Cloud Logging

Cloud Logging launches several usability features for effective troubleshooting. Learn more in this blog post.

BigQuery

BigQuery is introducing new SQL capabilities for improved analytics flexibility, data quality, and security. Some examples include schema support for Flexible column name, Authorized store procedures, ANY_VALUE (HAVING) also known as MAX_BY and MIN_BY, and many more. Check out full details here.

Community Security Analytics (CSA)

Community Security Analytics (CSA) can now be deployed via Dataform to help you analyze your Google Cloud security logs. Dataform simplifies deploying and operating CSA on BigQuery, with significant performance gains and cost savings. Learn more why and how to deploy CSA with Dataform in this blog post.

Deliver trusted insights with Dataplex data profiling and automatic data quality

Dataplex data profiling and AutoDQ are powerful new features that can help organizations improve their data quality and build more accurate and reliable insights and models. These features and now Generally Available.

Bring AI to Looker with the Machine Learning Accelerator

This easy-to-install extension allows business users to train, evaluate, and predict with machine learning models right in the Looker interface.