Open in app

Sign In

Write

Sign In

Amogh Margoor
Amogh Margoor

40 Followers

Home

About

Nov 30, 2021

Optimising memory for Aggregates and Join operators in Apache Impala.

Analytical SQL workloads use aggregates and joins heavily. Hence, optimising such operators for both performance and efficiency in analytical engines like Apache Impala can be very beneficial to users. …

Big Data

7 min read

Optimising memory for Aggregates and Join operators in Apache Impala.
Optimising memory for Aggregates and Join operators in Apache Impala.
Big Data

7 min read


Sep 15, 2020

Enabling Spark SQL MERGE via optimized ACID Data Source v0.6.0 | Qubole

Amogh Margoor, Sourabh Goyal, Vikram Agrawal Originally written by us here: https://www.qubole.com/tech-blog/enabling-spark-sql-merge-via-optimized-acid-data-source-v0-6-0/. Importing my technical blogs to this space. We are pleased to announce the 0.6.0 release of the ACID Data source for Apache Spark. This release should further empower Data lake users in enterprises to enable ACID support for…

6 min read

Enabling Spark SQL MERGE via optimized ACID Data Source v0.6.0 | Qubole
Enabling Spark SQL MERGE via optimized ACID Data Source v0.6.0 | Qubole

6 min read


Nov 9, 2019

Rust performance

As a part of thought experiment to create super fast Data processing engine at Petabyte scale, Programming language was one of the major consideration. I asked around my friends working/researching in the field of PL/compilers. Lot of them were quite excited about Rust and hence I started exploring it. While…

Programming

2 min read

Programming

2 min read


Jun 11, 2018

SQL Join Optimizations in Qubole Presto

Amogh Margoor, Rajat Venkatesh PS: Originally published on March 7, 2018: https://medium.com/qubole-engineering/sql-join-optimizations-in-qubole-presto-3ced3dc75275. I am moving my blogs to this space. SQL Joins are a common and critical component of interactive SQL workloads. …

7 min read

7 min read


Aug 15, 2017

Insights and Recommendations for Hive, Presto and Spark SQL Engine Configuration | Qubole

Amogh Margoor, Rajat Venkatesh, Dr Prasad M. Deshpande PS: Originally published by me on August15, 2017 here: https://www.qubole.com/blog/insights-recommendations-hive-presto-sparksql-engine-configuration/. Importing my technical blogs to this space. Execution engines like M/R, Tez, Presto, and Spark provide a set of knobs or configuration parameters that control the behavior of the execution engine. In…

9 min read

Insights and Recommendations for Hive, Presto and Spark SQL Engine Configuration | Qubole
Insights and Recommendations for Hive, Presto and Spark SQL Engine Configuration | Qubole

9 min read


Jun 19, 2017

Automatic Statistics Collection For Better Query Performance | Qubole

Amogh Margoor, Goden Yao Presto, Apache Spark, and Apache Hive can generate more efficient query plans with table statistics. For example, Spark, as of version 2.1.1, will perform broadcast joins only if the table size is available in the table statistics stored in the Hive Metastore (see spark.sql.autoBroadcastJoinThreshold). Broadcast joins…

4 min read

Automatic Statistics Collection For Better Query Performance | Qubole
Automatic Statistics Collection For Better Query Performance | Qubole

4 min read


Feb 8, 2016

Optimizing S3 Bulk Listings for Performant Hive Queries | Qubole

Introduction — We previously wrote about the optimizations we made to optimize Hadoop and Hive on S3. Since then, we’ve applied those same changes across the rest of our Big Data analytics offerings, including Spark and Presto. …

6 min read

Optimizing S3 Bulk Listings for Performant Hive Queries | Qubole
Optimizing S3 Bulk Listings for Performant Hive Queries | Qubole

6 min read

Amogh Margoor

Amogh Margoor

40 Followers
Following
  • Christine Fair

    Christine Fair

  • Jaemi Bremner

    Jaemi Bremner

  • Ashish Dubey

    Ashish Dubey

  • Mohan Krishnamurthy, Ph.D

    Mohan Krishnamurthy, Ph.D

  • Qubole Engineering

    Qubole Engineering

See all (41)

Help

Status

Writers

Blog

Careers

Privacy

Terms

About

Text to speech