Coralogix can be configured to automatically and dynamically archive logs to an S3 bucket. This saves Coralogix customers money, but of course there are times when the data needs to be reindexed. This operation counts the reindexed logs against the daily quota. Many times customers would like to search and focus on the exact logs to be reindexed or even query the logs outside of Coralogix all together.

Multi-cloud has become a hot-button topic these days, with a lot of cloud experts taking a strong stance against the practice. There are exceptions and legitimate reasons why you might choose a multi-cloud strategy (e.g., you sell to customers who restrict you from hosting their data in AWS). Extenuating circumstances aside, there are far more reasons why multi-cloud isn’t a good idea.

The hard part about application modernization isn’t determining whether or not it’s a good idea. It almost certainly is. Rather, the hard part is figuring out how to get started: Which applications you should focus on first, and what’s the best approach to modernizing them. Fully refactoring 1,000 applications from monolith into microservices, for example, might seem like the right goal, but it might also take the rest of this century.

A cloud-first strategy can feel like a big jump from traditional setups. One of the benefits of a hybrid or on-premises strategy is you feel like you’re in control. You and your team know where your critical servers live. You can touch them. Your team understands your security processes, and you can easily verify security personnel follow them. Those are all significant benefits. However, a growing number of software teams are choosing to move to cloud-first strategies.

Serverless has been gaining more and more traction over the last few years. The global serverless architecture market was estimated at $3.01 billion in 2017 and is expected to hit $21.99 billion by 2025. The number is reflected in the increasing amount of enterprises starting to look for ways of decoupling their current monolithic architectures and migrating their stack to serverless. Read more about the popular enterprise use cases for AWS Lambda.

At Illuminate, Genesys Cloud principal architect Kal Patel shared how they use Sumo Logic, how they manage it at scale, and some of the lessons they’ve learned in the last year.

Data sources ingest data in different sizes and shapes across on-premises and in the cloud, including product data, historical customer behaviour data, and user data. Enterprise could store these data in data storage services like Azure Blob store, an on-premises SQL Server, Azure SQL Database, and many more. This blog will highlight how users can define pipelines to migrate the unstructured data from different data stores to structured data using the Azure ETL tool, Azure Data Factory.

In a relatively short amount of time, Kubernetes has evolved from an internal container orchestration tool at Google to the most important cloud-native technology across the world. Today’s enterprises are adopting Kubernetes to speed up the development process and build and deploy modern applications at scale, and they’re doing so at an incredible pace.

Monitoring vs observability – is there even a difference and is your monitoring system observable? Observability has gained a lot of popularity in recent years. Modern DevOps paradigms encourage building robust applications by incorporating automation, Infrastructure as Code, and agile development. To assess the health and “robustness” of IT systems, engineering teams typically use logs, metrics, and traces, which are used by various developer tools to facilitate observability.

Even before new hybrid workforce models, many companies already moved a lot of services to the cloud. COVID-19 digital transformation strategies instantly increased the number of access points and endpoints. This led to a rapid increase in event log data followed by all kinds of other issues -- performance, availability, security, and ultimately increased IT costs amongst other things. A centralized log management solution for your cloud environment can help you manage the above and more.

In this article we explore the basics of monitoring Amazon Web Services (AWS) by feeding metrics to Grafana through Hosted Graphite’s agent and also through Hosted Graphite’s AWS add-on. This will allow us to monitor metrics from applications and servers hosted in AWS with clarity and depth. This article assumes you have created a Hosted Graphite account.

Although it is no surprise that Gartner is forecasting that overall public cloud consumption will grow by 18%, hitting $304.9 billion in 2021, Gartner’s research also indicates that the vast majority of organizations overshoot their cloud infrastructure budgets due to a lack of cost optimization approaches.

Cloud environments are susceptible to security issues. A big contributor is misconfigured resources. Misconfigured S3 buckets are one example of a security risk that could expose your organization’s sensitive data to bad actors. Policies and regular enforcement of best practices are key to reducing this security risk. However, manually checking and enforcing security is time-consuming and can fall behind with all the demands a busy DevOps team faces every day.

Release management is a topic that leaders in DevOps teams should be concerned with as organizations move toward implementing systems of automated continuous deployment. The practice will make your organization more efficient, but how do you implement it? Modernizing your infrastructure for the cloud is essential to distributing trusted releases globally. Many enterprises choose AWS as their platform, and seek to use AWS edge services to distribute their production applications to nearby end-points.

Here at CloudZero, we work with some of the top software-driven companies out there. Like us, they’re building their products on Amazon Web Services (AWS), along with whatever best-of-breed providers meet their needs. It’s no secret that in recent years, Snowflake has seen — well, some serious success. For many companies, including CloudZero, they're the data warehouse provider of choice — and an essential component of delivering their products.

So you've set up a Google Cloud Logging sink along with a Dataflow pipeline and are happily ingesting these events into your Splunk infrastructure — great! But now what? How do you start to get meaningful insights from this data? In this blog post, I'll share eight useful signals hiding within Google Cloud audit logs that will help you uncover meaningful insights. You'll learn how to detect: Finally, we’ll wrap up with a simple dashboard that captures all these queries in one place.

In the reliability era, many services are migrating from in-house servers to the cloud. The cloud model allows your service to capitalize on the benefits of large hosting providers such as AWS, Microsoft Azure, or Google Cloud. These servers can be more reliable than in-house servers for reasons including: However, as with all things, cloud providers present their own risks and challenges as well. Teams will want to take advantage of the benefits while accounting for these limitations.

Even though January 1 has passed, it’s not too late to commit to a New Year’s resolution. “Right–sizing” might sound like the latest fad diet that promises easy weight loss results. But you didn’t come to this particular blog because (like me) your pandemic diet sometimes consisted of nothing more than take-out and never-ending snacks.

The benefits for innovation with Kubernetes are clear: it can allow small teams to deliver more value, more rapidly. However, cost discussions around Kubernetes — and Kubernetes cost management — can be difficult. You have disposable and replaceable compute resources constantly coming and going, on a range of types of infrastructure. Yet at the end of the month, you just get a billing line item for EKS cost and a bunch of EC2 instances.

In this handbook, we’ll explain the AWS Step Functions Input and Output manipulation. There’s plenty to talk about AWS Step Functions. There are numerous articles available online talking about AWS Step Functions ever since Step Functions were introduced in 2016. Most of these articles might make you think that Step Functions are actually an extension of the Lambda function, allowing you to combine several Lambda functions to call each other.

At Google Cloud, we strive to bring Site Reliability Engineering (SRE) culture to our customers not only through training on organizational best practices, but also with the tools you need to run successful cloud services. Part and parcel of that is comprehensive observability tooling—logging, monitoring, tracing, profiling and debugging—which can help you troubleshoot production issues faster, increase release velocity and improve service reliability.

It has been nearly a year since Amazon Web Services (AWS) first rolled out its new savings plans. AWS Savings Plans were created to help manage cloud costs; however, they are best used as part of a larger strategy focused around cloud cost intelligence. (Keep reading for more on that!)

Although AWS Lambda is a blessing from the infrastructure perspective, while using it, we still have to face perhaps the least-wanted part of software development: debugging. In order to fix issues, we need to know what is causing them. In AWS Lambda that can be a curse. But we have a solution that could save you dozens of hours of time. TL;DR: Dashbird offers a shortcut to everything presented in this article.

The Splunk Deep Learning Toolkit (DLTK) is a very powerful tool that allows you to offload compute resources to external container environments. Additionally, you can use GPU or SPARK environments. In last Splunk blog post, The Power of Deep Learning Analytics and GPU Acceleration, you can learn more about building a GPU-based environment. Splunk DLTK supports Docker as well as Kubernetes and OpenShift as container environments.

Azure IoT Edge is a Microsoft Azure service that allows you to run containerized workloads on IoT devices. With IoT Edge and Azure IoT Hub, Azure’s device-management platform, organizations across science, manufacturing, energy production, and other industries can provision their IoT devices and workloads at the edge of their cloud networks for immediate in-unit computing, a necessity when running AI algorithms or parsing large datasets directly on IoT devices.

The adoption of cloud-based solutions has become increasingly common. The proof for this is evident – according to Gartner, Inc., the worldwide public cloud services market is expected to grow by 6.3% in 2020, up to a staggering $257.9 billion in value. The Flexera 2020 State of the Cloud Report, released on April 28, 2020, states that more than 90% of respondents have adopted cloud computing, with the top three cloud service providers being – AWS, Azure, and Google Cloud Platform.

As companies are increasingly built in the cloud, cost management has become a major issue for many engineering teams. While cloud providers like AWS offer flexibility and easy scaling, cloud costs can be opaque and difficult to track. As a result, many companies that rely on the cloud are adopting cloud cost management strategies to understand and manage the charges associated with their cloud-based technology, and maximize cloud usage and efficiency.

If you’re a product on the web, you have less than a second to impress your customers. Web application performance is crucial to the success of online products. Using a Content Delivery Network (CDN) like Cloudflare can provide just the edge you need.

Yup – we've now got a health explorer for Azure VMs! Health Explorer… now why does that sound familiar? You’ve probably guessed it by now, but here’s another hint: Hmm…now where have I seen that before? Right – in SCOM!

Recently, I saw a video from a really great developer and YouTuber, Ben Awad, where he discussed Serverless not make any sense. Even though I really enjoyed the video, I am not sure if the author’s points about serverless are entirely valid, and I want to discuss them in this article.

Centralized Log Management offers the visibility you need to optimize your cloud usage to keep infrastructure costs down. Cloud-first infrastructures are the future of modern business operations. As organizations like Google and Twitter announce long-term plans for enabling a remote workforce, maintaining a competitive business model includes scaled cloud services adoption. While the cloud offers scalability that can save money with pay-as-you-need services, managing the costs is challenging.

Spot by NetApp’s Ocean continuously optimizes Kubernetes clusters with a wide feature-set tackling different aspects of running and managing Kubernetes containers in a cloud environment. One such aspect are the container resource requests defined in the cluster (upon which Ocean intelligently bin-packs pods on the underlying cloud VMs). Incorrect assumptions regarding the CPU and Memory required for an application, can incur unnecessary and costly cloud infrastructure waste.

Work, education, and even many of our leisure activities have all moved on-line at an incredible pace due to current social distancing mandates. The digital backbone of the Internet and the SaaS services that drive our personal and professional lives are now foundational. Ensuring that these systems are operating optimally and securely is of paramount importance.

Microsoft Azure has become the go-to cloud computing service for over 95% of Fortune 500 companies—and for good reason. Azure’s flexible and scalable cloud environment offers a secure off-premises solution for your business IT infrastructure, without the need to manage physical servers in your changing IT infrastructure. Azure allows you to manage servers, databases, applications, and more, all from one of Microsoft’s secure global cloud storage sites.

AWS AppSync is a fully managed GraphQL service that makes it easy for you to build scalable and performant GraphQL APIs without having to manage any infrastructure! With AppSync, you get a lot of capabilities out of the box. Such as the ability to integrate directly with DynamoDB, ElasticSearch, Aurora Serverless, and Lambda. AppSync also supports both per-request as well as per-resolver caching and has built-in integration with CloudWatch and X-Ray.

At first glance, all serverless monitoring services seem similar and aim to solve the same problems. However, in Dashbird, we have made decisions that fundamentally differentiate us from our competitors since day one. Over time, those differences have magnified and we have found increasing confirmation and confidence in our approach. Dashbird product strategy is based on three core pillars.

AWS Lambda is a serverless compute service that runs your code in response to events and automatically manages the underlying compute resources for you. The code that runs on the AWS Lambda service is called Lambda functions, and the events the functions respond to are called triggers. Lambda functions are very useful for log collection (think of log arrival as a trigger), and Coralogix makes extensive use of them in its AWS integrations.

Do you have an application that’s a little… sluggish? Cloud Profiler, Google Cloud’s continuous application profiling tool, can quickly find poor performing code that slows your app performance and drives up your compute bill. In fact, by helping you find the source of memory leaks and other errors, Profiler has helped some of Google Cloud’s largest accounts reduce their CPU consumption by double-digit percentage points.

If you’re spending more than you expected on your Kubernetes deployment, you’re not alone. Many Kubernetes operators are experiencing higher Kubernetes costs than what they had predicted. That’s because, like many aspects of Kubernetes, identifying how to manage or lower costs can be challenging. In this article, we provide 5 essential tips for how you can achieve a more cost-efficient Kubernetes deployment.

Logz.io has recently launched its Smart Tiering solution, which gives you the flexibility to place data on different tiers to optimize cost, performance and availability. Our mission has been to make Smart Tiering a multi-cloud and multi-region service. As part of this launch, we are glad to announce that the Historical Tier now supports Microsoft Azure Blob Storage, alongside AWS S3.

Every year, Redgate’s State of Database Monitoring Report reveals how businesses and organizations are monitoring their database estates. Are they using in-house or third-party monitoring tools? Who has access to the data? What are their biggest challenges? The thousands of responses to the survey behind the report offer the answers, and also provide an opportunity to dive deeper and examine those issues at an industry sector level.

One problem that pops up quite frequently when people try to build serverless applications with AWS API Gateway and AWS Lambda is Execution failed due to configuration error: Malformed Lambda proxy response. There is nothing worse than generic error messages that don’t tell you anything you need to fix the problem, right? And AWS isn’t particularly known for its error message design, if you can even call it that, let alone for giving you the means of fixing the problem.

As cloud infrastructure grows and develops, reliable and safe management of containers across multiple cloud providers grows increasingly important - accelerating the adoption of Kubernetes (K8s). Orchestration technologies like Kubernetes (K8s) automate the deployment and scaling of containers, and they also ensure the reliability of applications and workloads running on containers.

One of Amazon Web Services’ (AWS) most well-known services is AWS DynamoDB. Some of AWS’s most notable customers use DynamoDB for their database needs – companies such as Netflix, The Pokemon Company, and Snapchat. DynamoDB is relatively simple to set up and configure, and it integrates well with many web-based applications. DynamoDB supports technology solutions in gaming, retail, bank and finance, and the software industry.

For quick, scalable, highly-available web services, few options compare to AWS Lambda. Just provide your code, add a little configuration, and you're done! In this article, Milap Neupane will introduce us to Lambda, show us how to get it working with Ruby and the Serverless Framework, and discuss reasons to use — or to not use! — Lambda in production.

With more and more businesses moving online, and homegrown entrepreneurs spinning up new online apps, they’re increasingly looking for an online development platform to help them easily build and deploy their sites.

There has been a significant amount of change in the IT industry over the past 25 years, from the integration of email, the move to laptops over desktops and the introduction and transition to cloud computing. However, nothing compares to witnessing the rapid change and innovation businesses have adopted throughout the pandemic, including ourselves. Agility, adapting to survive

In this article, we are going to see how we can integrate Google Cloud with Grafana. We will integrate Google Cloud SQL with Grafana and plot the metrics on Grafana. We will also look at how we can use Google Stackdriver as the data source in Grafana to expose the metrics of Google Cloud VM’s and platforms. To use Grafana immediately, we will be using Hosted Grafana by MetricFire.

Using the cloud reduces on-premises infrastructure costs and related maintenance. Instead of deploying more servers, storage, and networking components to your own datacenter, you are now deploying these as cloud resources. Using the cloud is supposed to reduce infrastructure and maintenance costs. However, deploying cloud resources also risks over-commissioning, under-usage, and keeping resources running that are not always needed or, even worse, no longer in use.

As an AWS Advanced Technology Partner with AWS Containers Competency, Tigera is thrilled to announce that Calico and Calico Enterprise are both now available as AWS Quick Starts. If you’re unfamiliar with the concept, an AWS Quick Start is a ready-to-use accelerator that fast-tracks deployments of key cloud workloads for AWS customers.

Should you run big data workloads on Kubernetes? Not so long ago, most people would have said certainly not. Among the reasons you might have heard: Although early adoption of Kubernetes was dominated by stateless services rather than data-centric applications, more and more teams responsible for big data platforms are now looking to adopt Kubernetes, especially those looking to deploy and operate big data workloads in the cloud.

As with many things in 2020, this year’s AWS re:Invent was quite different from any previous iterations. For starters, instead of a week of live talks, face-to-face sessions, and a room full of booths, this year the event was fully online and stretched out for three weeks. As sponsors of this year’s event, we were excited to participate and continue to make an impact on the AWS community.

In this article, we’ll be discussing everything you need to know about the basics of AWS Lambda error handling and some popular methods using StepFunctions and X-Ray. Regardless if you’re an AWS Lambda expert or if you’re a new Lambda user, there’s always something new to learn.

For developers and programmers, Amazon Web Services (AWS) offers many benefits. It gives you access to the computing and DevOps tools you need at the press of a button — which helps you get products out the door fast. However, it can be challenging to control your costs and identify waste. In this comprehensive guide, we will examine some practical steps you can take for AWS cost optimization.

In our last blog, "What's New in Splunk Cloud: Part 1," we reviewed a host of new Splunk Cloud features that we have delivered through our accelerated releases since the beginning of 2020. A large part of this effort focused on empowering Splunk Cloud admins and making their experience as self-service as possible. In this blog, we will examine our latest effort to continue this empowerment: Splunk Cloud’s Admin Configuration Service (ACS).

Elastic Cloud puts the power of the Elastic Stack in your hands within minutes. Whether you’re trying to add search capabilities with Elastic Enterprise Search, monitor critical systems and applications with Elastic Observability, or protect your organization from cyber threats with Elastic Security, taking the first step is easy.

Among all the new features and services that AWS announced during the re:Invent 2020, my favorites were definitely the AWS Lambda updates. And there were many! For example, your code execution is no longer rounded up to the nearest 100ms of duration for billing — you are now billed on a per millisecond. On top of that, AWS increased the Lambda’s memory capacity to 10 GB, and correspondingly the CPU capacity up to 6 vCPUs.

There is no doubt that organizations have a difficult task to address the current requirements for building and managing APIs, especially if they don’t use tools like API Management. APIs require up-to-date and strong documentation, increased security levels, comprehensive testing, routine versioning, and high-reliability to be appropriately utilized.

When you’re troubleshooting an app or a deployment, every second counts! Cloud Logging helps you troubleshoot by aggregating logs from across Google Cloud, on-premises or other clouds, indexing, aggregating logs into metrics, scanning for unique errors with Error Reporting and making logs available for search, all in less than a minute. And now, we’ve built two new features for streaming logs to give you even fresher insights from your logs data.

Amazon DocumentDB is a managed MongoDB-compatible database service provided by AWS. It provides the database in clusters, with multiple instances, for high-availablity. To help with cost-reduction strategies, AWS allows DocumentDB clusters to be stopped and restarted. While the cluster is stopped, you’re not charged. So it’s a great candidate to shut off overnight and on weekends if it’s not needed.

When we talk about going Cloud-Native we often think about Microservices as well, probably because that’s where the actual innovation, time, cost-saving, and better utilization of resources happen. And CloudHedge is keenly focused on helping enterprises to bring the benefits of containers to traditional legacy business applications that are run on a widely used operating system: AIX!

Happy New Year everyone! We are thrilled to be starting 2021 with some exciting news. Come February 2nd, we’ll be kicking off Azure Management Talk, a bite-sized webinar series with a focus on all things Azure management. Azure is fast-evolving, and often, it can get quite complicated. With so many things to learn and not enough time, the huge swathes of learning resources available online can quickly get overwhelming.

Increasingly, we are seeing on-prem workloads being moved onto the cloud. Elasticsearch has been around for many years with our users and customers typically managing it themselves on-prem. Elasticsearch Service on Elastic Cloud — our managed Elasticsearch service that runs on Amazon Web Services (AWS), Google Cloud, and Microsoft Azure across many different regions, is the best way to consume the Elastic Stack and our solutions for enterprise search, observability, and security.

Continuous integration and deployment (CI/CD) processes are becoming the standard for many organizations operating in the cloud—including many Spot by NetApp customers—with Jenkins as a core tool for building, testing and deploying code through CI/CD pipelines. Spot has had an integration with Jenkins since our early days, allowing our users to configure Elastigroups managed by Jenkins and deploy jobs running on spot instances.

In December, AWS announced the new VPC Reachability Analyzer tool. Using this tool, AWS can analyse the paths within your VPC to determine whether the various components in your VPC can communicate with each other or not.

We all know that debugging serverless is time-consuming and hard and that AWS Console doesn’t make it much easier. CloudWatch isn’t quite known for its ease of use. Why? Well to start with, it has suboptimal search features, logs scattered across multiple buckets and groups, little visualization capability, and no structure of Lambda function invocations.

One of the keys to CloudWatch’s success is its no bang, no buck billing system. The pricing structure has been designed from the outset to ensure that CloudWatch users only pay for what they actually use. In addition, the CloudWatch Free Tier allows first time users to test the waters without shelling out. The downside of this flexibility and adaptability is complexity.