ChartPlayer v0.1.20 is now available:

https://github.com/mikeoliphant/ChartPlayer/releases/latest

Changes: 

• Better chord detection
• Integrated chromatic tuner
• Drag on timeline for fine scrolling
• Timeline UI tweaks
• Added rewind button and auto-rewind at end of song

 

 

 

Yeah this awesome.
Lower Latency than when I'm using RS, flexible and easy playhead adjustment
Visually easier to identify notes on the fretboard imho
It would be nice to have a looping function similar to riff repeater in RS2014, just not as disruptive
If you could simply highlight or double click a section to loop, that would improve practicing tough sections like solos by leaps and bounds. Thanks for letting us test it out. 
 

18 hours ago, Zoltan Pepper said:

It would be nice to have a looping function similar to riff repeater in RS2014, just not as disruptive

Minimal looping is already in there. Use "[" to mark the start of a loop and "]" to mark the end. There is certainly room for UI improvement, but you can at least do it...

19 hours ago, Zoltan Pepper said:

If you could simply highlight or double click a section to loop, that would improve practicing tough sections like solos by leaps and bounds. Thanks for letting us test it out. 

Double-click on a section to set it as the loop region is a good idea - I just added that.

 

Thank you very much for the excellent work you've been doing. What you've done so far is already great. If you can, try adding an option to switch between the standard Rocksmith view and a timeline with the strings, like tablature, like in Rocksmith Plus. The vocalization is user-friendly and easy to learn. And if you can, also add the functionality of the song simply continuing to play and rocking the right note if we're learning a section, also like in Rocksmith Plus. It makes learning easier. But as I said, the work is already great. If you can, your VST will be great. Big hug from a Brazilian guitarist.

 

This is so cool, I wish I saw it earlier!
I started making something like this last year, but in Godot.
I did a lot of experimentation in note/chord detection, and had pretty solid results in the end.

Here is some of the code involved.
NOTE: Before these steps, the autocorrelation is normalized by dividing every element by the first.

 

Single notes: Check that the correlation in the relevant bin is >0.3 AND that the correlation is lower for BOTH adjacent semitones/frets.
The detected peak correlation is not actually used here.

Chords: Sum up the correlations of the relevant bins for all chord notes, check that the sum is >0.3 AND that the peak bin corresponds to ANY note in the chord.
The 0.97 to 1.03 range is roughly half a semitone, the tolerance around the center of the peak bin (which won't perfectly align with a written note).

# Note hit detection
var hit = false
for note in get_tree().get_nodes_in_group("notes"):
    if not note.hit \
    and Global.time > note.time - Global.HIT_TOLERANCE \
    and Global.time < note.time + note.sustain + Global.HIT_TOLERANCE:
        var freq      = Convert.fret_to_freq(note.string, note.fret    )
        var freq_up   = Convert.fret_to_freq(note.string, note.fret + 1)
        var freq_down = Convert.fret_to_freq(note.string, note.fret - 1)
        var corr      = $Audio/Mic.correlation_at_frequency(freq     )
        var corr_up   = $Audio/Mic.correlation_at_frequency(freq_up  )
        var corr_down = $Audio/Mic.correlation_at_frequency(freq_down)
        if corr > 0.3 and corr > corr_up and corr > corr_down:
            note.hit = true
            hit = true
for chord in get_tree().get_nodes_in_group("chords"):
    if not chord.hit:
        var has_peak := false
        var chord_corr := 0.0
        for note in chord.note_nodes:
            if  Global.time > note.time - Global.HIT_TOLERANCE \
            and Global.time < note.time + note.sustain + Global.HIT_TOLERANCE:
                var freq = Convert.fret_to_freq(note.string, note.fret)
                var corr = $Audio/Mic.correlation_at_frequency(freq)
                chord_corr += corr
                var peak_ratio = freq / Global.peak_frequency
                if peak_ratio > 0.97 and peak_ratio < 1.03:
                    has_peak = true
        if chord_corr > 0.3 and has_peak:
            for note in chord.note_nodes:
                note.hit = true
            chord.hit = true
            hit = true

The peak bin is simply determined by the largest correlation.
However, I am only looking after the first rising zero-crossing and only in the region <1600Hz.
(I don't entirely remember why I check zero-crossing instead of simply increasing, but I know it was deliberate)

# Return the peak frequency of the autocorrelation
func _peak_freq(correlation : Array, sample_rate : float) -> float:
    # Find the first negative-to-positive zero crossing
    var start_index := 0
    for i in range(1, correlation.size()):
        if correlation[i-1] < 0 and correlation[i] >= 0:
            start_index = i
            break
    # Find the maximum value after the zero crossing
    var max_value := 0.0
    var max_index := 0
    for i in range(start_index, correlation.size()):
        # Skip frequencies above 1600 Hz
        if sample_rate / i >= 1600:
            continue
        var curr = correlation[i]
        if curr > max_value:
            max_value = curr
            max_index = i
    if max_index > 0:
        return sample_rate / max_index
    else:
        return 0.0

I removed checking of other octaves since I (empirically) found it to be unnecessary (to my surprise).
I also removed windowing from the FFT since I did not find any improvement with it (also to my surprise).

Another thing to note is that my underlying C++ spectrum analyzer can switch from using FFT for autocorrelation to using a "lag" method.
FFT is O(N log N) while lag is O(N^2), but I found them to cross over in computation time around N=2048 (iirc).
Ultimately, that means the FFT is the same or better for any realistic sample size, but it's an interesting consideration.
(and a good FFT library is going to be faster than my implementation)

My FFT is currently done using N audio samples, but padded with N zeros.
(so FFT size is 2*N and we assume N is already a power of 2)
This is basically a cheap interpolation that allows a lower sample size to have finer bins.

 

My chord detection effectively detects any subset of the chord.
If you play one string, its correlation will be high while the correlations for the other strings will be near zero.
If you play multiple strings, the correlations naturally reduce each other such that the sum of them is similar to playing just one string.
A wrong note will pull down the correlations of any correct notes, bringing the sum under the hit/miss threshold.

I might be misreading your code (NoteDetector.NoteDetect method), but it looks like you require ALL notes of the chord to be played for it to count?

I am deliberately allowing for single notes to count as chords, since in Rocksmith I very often skip strings either because I'm sight-reading too fast or because I don't feel like putting my hand in an uncomfortable shape.

 

Hoping some part of this ramble can help with your note/chord detection!

9 hours ago, blockchaaain said:

I might be misreading your code (NoteDetector.NoteDetect method), but it looks like you require ALL notes of the chord to be played for it to count?

Yes, it requires all of the pitches to be present. It isn't octave-specific, though, so you can get away with missing out notes in the chord that duplicate a pitch.

I only use auto-correlation for single note detection. It works very well for that, but not so great for chords. For chords I'm using a more general analysis of spectrum peaks.

9 hours ago, blockchaaain said:

I also removed windowing from the FFT since I did not find any improvement with it (also to my surprise).

Yes, windowing is important for looking at the spectrum, but you don't want it for auto-correlation. Similarly, zero-padding is important for auto-correlation but not for getting the spectrum.

Just a note here saying that I often had the feeling Rocksmith itself does something similar to what  @ blockchaaain  says, if you play an incomplete chord it treats it as correct, but if you add an extra note it treats it as wrong. I don't really know but I often felt it was the case...

Hi ladron, does your tool currently support multiplayer? If not, are there any plans to add it in the future? That would make it the ultimate pro Guitar Hero experience
 

5 hours ago, andrezzz7 said:

Hi ladron, does your tool currently support multiplayer? If not, are there any plans to add it in the future? That would make it the ultimate pro Guitar Hero experience

No multiplayer support yet, but it is something I am likely to look at adding at some point.

ChartPlayer v0.1.21 is now available:

https://github.com/mikeoliphant/ChartPlayer/releases/latest

Changes: 

 

•     Better detection of single notes during fast string transitions
•     Double-click song section to set as loop region
•     Reduced memory allocations in song player
•     Don't respond to mouse clicks outside of active window area
•     More standard drop tuning naming
•     Display song difficulty and song length in song list
•     Fixed bug with song accuracy formatting

Note: to get song difficulty data (as well as song length data for Rock Band Songs) you will need to re-convert using the latest ChartConverter.

 

 

 

ChartPlayer v0.1.22 is now available:

https://github.com/mikeoliphant/ChartPlayer/releases/latest

Changes:

•     Fixed regression with accuracy scoring
•     Fixed bug causing incorrect drum song section difficulty calculation

 

 

ChartPlayer v0.1.23 is now available:

https://github.com/mikeoliphant/ChartPlayer/releases/latest

Changes: 

• Better detection of single note slides
• Fixed note detection for songs with a capo
• Added visual capo indicator
• Exclude "preview.ogg" and "crowd.ogg" from Rock Band audio mixes
• Shifted drum timeline current time offset

 

..hey Ive got some official instrumentals of songs..id like to swap & inject them into existing charts

instead of looking for ways to take the vocals out of the charts..maybe just swap w/ instrumentals?

this should be easy to do w/ ChartPlayer right? can someone pls briefly explain that process?

15 hours ago, brickinthewall said:

..hey Ive got some official instrumentals of songs..id like to swap & inject them into existing charts

instead of looking for ways to take the vocals out of the charts..maybe just swap w/ instrumentals?

If I am understanding you correctly, this would require re-charting. You could start with the original chart, but it would need to be time aligned to the instrumental version of the song.

2 hours ago, ladron said:

If I am understanding you correctly, this would require re-charting. You could start with the original chart, but it would need to be time aligned to the instrumental version of the song.

ok..the timing alignment..re-charting..I'll figure it all out..thx

oh & your Github bro..JazzStandards Chord Data..yeeesss let's go!

but I'll try 3 tracks..1: Neural DSP..2: Instrumental track..3: ChartPlayer

if I play the Instrumental & ChartPlayer simulatenously..but mute the PSARC song

only read/play charts..timing would be too off? either way having this in Ableton wow💥🚀🪐

Tested ChartPlayer with a few songs and I love it! Thx for creating this! ❤️

The only difference I note right now is, on bass, when songs have a tuning under C in Rocksmith, I got that mod that makes shifts the colors Red, Yellow, Blue down one string, and it adds a Cyan like color for the at the E string position. That really helps when playing bass on 5 string bass songs!

Other then that, I'll keep testing this, but it will definitely replace Rocksmith for me!

14 hours ago, Matoto951 said:

The only difference I note right now is, on bass, when songs have a tuning under C in Rocksmith, I got that mod that makes shifts the colors Red, Yellow, Blue down one string, and it adds a Cyan like color for the at the E string position. 

That seems reasonable. I've added an issue to track it:

https://github.com/mikeoliphant/ChartPlayer/issues/34

 ChartPlayer v0.1.24 is now available:

https://github.com/mikeoliphant/ChartPlayer/releases/latest

Changes:  

•     Fixed note detection on songs offset from A440
•     Use a low green string when playing in B std or lower
•     Use the first tuning for an instrument to show in the song list
•     Made active drum display hits bigger
•     Only save the current instrument stats
•     Catch and log any errors scanning songs

 

 

 

im not sure if im confused or dont see the link but this is only available for linux and mac? what if i wanted to use it on windows?

3 hours ago, Jabelnyc.g said:

im not sure if im confused or dont see the link but this is only available for linux and mac? what if i wanted to use it on windows?

The "ChartPlayerVST3Plugin.zip" download is the Windows VST plugin.