04/3/19

Distributed Transactional Locks

As I explained in a previous blog post, sometimes MVCC is not sufficient and an operation needs to block out all other concurrent modifications. NuoDB is able to lock three types of lockable resources: tables, schemas, and sequences. A resource can either be locked in SHARED mode (which still allows record modification, but no metadata modification) or EXCLUSIVE which prevents any concurrent modification.

EXCLUSIVE access is required for DDL that visits all records (various index operations for example) and distributed concurrent access is not possible. Exclusive access can also be used by operations that need to work around MVCC write skew anomalies.

SHARED locks need to be fast, consume as little memory as possible, involve no additional nodes in the cluster, and cause no additional messaging.

EXCLUSIVE locks, on the other hand, need to pay the cost.

MVCC uses row locks to serialise updates to a single record. Both transactional locks and row locks participate in the same deadlock detection process, but are otherwise independent. The following table explains the interaction between transactional locks and MVCC row locks. If you are interested in how row locks work, I recommend reading our MVCC blog series.

Row Locks (DML) / Transactional LocksSharedExclusive
selectNo ConflictNo Conflict
insert, update, delete, select for updateNo ConflictConflict
Continue reading
If you enjoyed this post, then make sure you subscribe to my Newsletter and/or Feed.

Facebooktwittergoogle_plusredditpinterestlinkedin
04/3/19

How Transaction Locks support Zero Overhead Distributed DML

Let us imagine a scenario that needs to prevent MVCC write skews…

One transaction increases the salary of everyone in a department by 10%; another transaction inserts a new employee with a salary X. Since the two transactions do not conflict, MVCC does not prevent either of them from committing. After both resolve, the overall salary in the department could be above the budget. To prevent a similar situation, the application developer might want to have exclusive access to the table.

NuoDB 3.2.2 exposes a new type of lock that guarantees exclusive access to a resource across the distributed cluster. We call them Transactional Locks and they are the underlying mechanism powering the new NuoDB LOCK statement.

Continue reading
If you enjoyed this post, then make sure you subscribe to my Newsletter and/or Feed.

Facebooktwittergoogle_plusredditpinterestlinkedin
09/24/18

What I discovered using a simple challenge during interviews

Can coders code?

When I was looking for a simple programming task that I can use to evaluate candidates, I had the following criteria in mind:

  • solvable within 10 min
  • does not require obscure algorithmic knowledge
  • has potential for further discussion
  • not language specific

I went through my list of simple problems (I only considered the easy ones) and I stumbled across Pairs. I instantly realized, that it is the perfect question. There is a brute force solution, the specification can be expanded, we can talk about sorting, hashing, hash collisions, Big-O notation and cases when the input does not fit in memory. Even Bloom Filters are a viable topic of discussion.

My first interview was planned and I was thrilled. I would use the challenge and it would be great! The first three candidates I exposed to the task failed horribly. They could not implement the brute force solution O(n^2) without bugs in the code. I was horrified. Is the industry really this bad? Can programmers really not program?

In the last 3 years I used this little coding task in almost 50 interviews and the failure rate is alarming. Only two people managed to solve it flawlessly. They both accepted the offer and I have been thrilled to work with them ever since. There have been others that got stuck in the weeds but nevertheless showed an ability to think. Kudos to those! I will expand on the definition of flawlessly below.

I adjust the -pass- criteria accordingly to the person’s experience (people who graduated a long time ago might not remember what Big-O is), title (did they code in the last job?), future job duties (are they supposed to code?) and the prowess with which they claim that they know how to program. Support engineers that have had limited exposure to python and shell are fine with something that resembles code. People that have been in the industry for a decade better be ready to go through at least 3 iterations of increasing complexity.

Before reading on, I suggest that you go and solve that challenge. I will wait….

Continue reading
If you enjoyed this post, then make sure you subscribe to my Newsletter and/or Feed.

Facebooktwittergoogle_plusredditpinterestlinkedin

09/22/18

HackerRank ‘The Power Sum’ Solution

Short Problem Definition:

Find the number of ways that a given integer, X , can be expressed as the sum of the Nth powers of unique, natural numbers.

For example, if X = 13 and N = 2, we have to find all combinations of unique squares adding up to 13. The only solution is 2^2 + 3^2.

Link

The Power Sum

Complexity:

time complexity is O(N!)

space complexity is O(1)

Execution:

This solution does not use DP or memoisation, but it does not time out. We know that it needs to explore all P! possible combinations where P is floor(sqrt(X, N)). Since maximum X is 1000, this should still compute in real time.

Keep in mind that all integers in the solution have to be unique. So for X = 3, the solution 1^2 + 1^2 + 1^2 is not acceptable.

Did I mention that I dislike both NP-Hard problems and recursion? No? So, now you know.

Solution:
def powerSum(X, N, current = 1):
    pw = pow(current, N)
    if pw > X:
        return 0
    elif pw == X:
        return 1
    else:
        return powerSum(X, N, current+1) + powerSum(X-pw, N, current+1)

If you enjoyed this post, then make sure you subscribe to my Newsletter and/or Feed.

Facebooktwittergoogle_plusredditpinterestlinkedin

09/21/18

HackerRank ‘Lily’s Homework’ Solution

Short Problem Definition:

Whenever George asks Lily to hang out, she’s busy doing homework. George wants to help her finish it faster, but he’s in over his head! Can you help George understand Lily’s homework so she can hang out with him?

Consider an array of m distinct integers, arr = [a[0], a[1], …, a[n-1]]. George can swap any two elements of the array any number of times. An array is beautiful if the sum of |a[i] – a[i-1] among 0 < i < n is minimal.

Link

Lily’s Homework

Complexity:

time complexity is O(N*log(N))

space complexity is O(N)

Execution:

Let us rephrase the problem to a sorting problem: Find the number of swaps to make the array sorted. We need to consider both ascending and descending arrays. The solution assumes no duplicates.

Solution:
def cntSwaps(arr):
    positions = sorted(list(enumerate(arr)), key=lambda e: e[1])
    swaps = 0
    
    for idx in xrange(len(arr)):
        while True:
            if (positions[idx][0] == idx):
                break
            else:
                swaps += 1
                swapped_idx = positions[idx][0]
                positions[idx], positions[swapped_idx] = positions[swapped_idx], positions[idx]
    
    return swaps

def lilysHomework(arr):
    return min(cntSwaps(arr), cntSwaps(arr[::-1]))

If you enjoyed this post, then make sure you subscribe to my Newsletter and/or Feed.

Facebooktwittergoogle_plusredditpinterestlinkedin

09/20/18

HackerRank ‘HackerRank Bear and Steady Gene’ Solution

Short Problem Definition:

A gene is represented as a string of length N (where  is divisible by 4), composed of the letters A, C, G, and T. It is considered to be steady if each of the four letters occurs exactly 1/4 times. For example, GACT and AAGGCCTT are both steady genes.

Link

Bear and Steady Gene

Complexity:

time complexity is O(N)

space complexity is O(1)

Execution:

A bit of trivia: I’ve looked at this problem a couple times 2 years ago and I could not figure it out. Sometimes it is good to take a long break 🙂

In the first pass I establish how many occurrences there are of each character. Then I establish how many are missing to make the string steady. Remember that the number of occurrences needs to be evenly distributed.

The second pass takes advantage of the caterpillar method. I am looking for the shortest string that contains all the extra characters. They need to be replaced by the missing characters. All others can just be replaced with the same character. The caterpillar extends as long as it does not hit one of the extra characters. It shortens if it contains more extra characters than are needed.

Solution:
def steadyGene(gene):
    min_length_string = len(gene)
    
    occurences = dict()
    occurences['A'] = 0
    occurences['G'] = 0
    occurences['C'] = 0
    occurences['T'] = 0
    
    expected = len(gene) // 4
    
    for g in gene:
        occurences[g] += 1
    
    for x in occurences:
        occurences[x] = max(0, occurences[x] - expected)
    
    if occurences['A'] == 0 and occurences['G'] == 0 and occurences['C'] == 0 and occurences['T'] == 0:
        return 0
    
    found = dict()
    found['A'] = 0
    found['G'] = 0
    found['C'] = 0
    found['T'] = 0
    
    tail = 0
    head = 0
    
    while head != len(gene):
        found[gene[head]] += 1
        if found['A'] >= occurences['A'] and \
        found['C'] >= occurences['C'] and \
        found['G'] >= occurences['G'] and \
        found['T'] >= occurences['T']:
            # this is a valid candidate
            min_length_string = min(min_length_string, head-tail+1)
            
            # try to shorten it
            while found[gene[tail]] > occurences[gene[tail]]:
                found[gene[tail]] -= 1
                tail += 1
                min_length_string = min(min_length_string, head-tail+1)
            
            
        head += 1
    
    return min_length_string

If you enjoyed this post, then make sure you subscribe to my Newsletter and/or Feed.

Facebooktwittergoogle_plusredditpinterestlinkedin

09/19/18

HackerRank ‘Sherlock and The Valid String’ Solution

Short Problem Definition:

Sherlock considers a string to be valid if all characters of the string appear the same number of times. It is also valid if he can remove just 1 character at 1 index in the string, and the remaining characters will occur the same number of times. Given a string , determine if it is valid. If so, return YES, otherwise return NO.

Link

Sherlock and The Valid String

Complexity:

time complexity is O(N)

space complexity is O(N)

Execution:

This is one of the easier medium problems. Create a character occurrence map. Then create an occurrence-occurrence map. If all the occurrences are the same (size is 1) the string is valid. If there is exactly one character that occurs exactly once, it is also valid. Otherwise invalid

Solution:
def isValid(S):
    char_map = Counter(S)
    char_occurence_map = Counter(char_map.values())

    if len(char_occurence_map) == 1:
        return True

    if len(char_occurence_map) == 2:
        for v in char_occurence_map.values():
            if v == 1:
                return True

    return False

If you enjoyed this post, then make sure you subscribe to my Newsletter and/or Feed.

Facebooktwittergoogle_plusredditpinterestlinkedin

09/18/18

HackerRank ‘String Construction’ Solution

Short Problem Definition:

Amanda has a string of lowercase letters that she wants to copy to a new string. She can perform the following operations with the given costs. She can perform them any number of times to construct a new string p:

  • Append a character to the end of string p at a cost of 1 dollar.
  • Choose any substring of p and append it to the end of  at no charge.
Link

String Construction

Complexity:

time complexity is O(N)

space complexity is O(N)

Execution:

The solution sounds too easy, but it is still very simple. A substring of length 1 is still a substring. Each character in the final string needs to be copied once for 1$. Each other occurrence of that string can be copied for 0$. Aka just count the number of distinct letters in the expected string.

Solution:
def stringConstruction(s):
    return len(set(s))

If you enjoyed this post, then make sure you subscribe to my Newsletter and/or Feed.

Facebooktwittergoogle_plusredditpinterestlinkedin

09/17/18

HackerRank ‘Weighted Uniform Strings’ Solution

Short Problem Definition:

A weighted string is a string of lowercase English letters where each letter has a weight. Character weights are 1 to  26 from a to z…

Link

Weighted Uniform String

Complexity:

time complexity is O(N)

space complexity is O(N)

Execution:

Parsing the string for every query is suboptimal, so I first preprocess the string. Now we know that uniform strings contain the same characters. A string can be of length 1. Do a single pass of the string and create all uniform substrings.

Solution:
def weightedUniformStrings(s, queries):
    weights = set()
    prev = -1
    length = 0
    for c in s:
        weight = ord(c) - ord('a') + 1
        weights.add(weight)
        if prev == c:
            length += 1
            weights.add(length*weight)
        else:
            prev = c
            length = 1
    
    rval = []
    for q in queries:
        if q in weights:
            rval.append("Yes")
        else:
            rval.append("No")
    return rval

If you enjoyed this post, then make sure you subscribe to my Newsletter and/or Feed.

Facebooktwittergoogle_plusredditpinterestlinkedin

09/16/18

HackerRank ‘HackerRank in a String!’ Solution

Short Problem Definition:

We say that a string contains the word hackerrank if a subsequence of its characters spell the word hackerrank. For example, if string s = haacckkerrannkk  it does contain hackerrank, but s = haacckkerannk does not. In the second case, the second r is missing. If we reorder the first string as , it no longer contains the subsequence due to ordering.

Link

HackerRank in a String!

Complexity:

time complexity is O(N)

space complexity is O(1)

Execution:

Keep two pointers. One to the expected string (needle) and one to the input string. If you find the needle in the haystack before you run out of characters, you are good.

Solution:
def hackerrankInString(s):
    needle = 'hackerrank'
    idx_in_needle = 0
    for c in s:
        if c == needle[idx_in_needle]:
            idx_in_needle += 1
        if idx_in_needle == len(needle):
            break
            
    if idx_in_needle == len(needle):
        return "YES"
    else: 
        return "NO"

If you enjoyed this post, then make sure you subscribe to my Newsletter and/or Feed.

Facebooktwittergoogle_plusredditpinterestlinkedin